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Information Rate in a Continuous Channel for 


Regular-Simplex Codes* 


C. A. STUTTT, MEMBER, IRE 


Summary—M equally probable symbols may be encoded as 
continuous waveforms whose sample values are the coordinates 
of the vertices of a regular simplex in N dimensions. These vertices 
or code points, therefore, are equally spaced, and the resulting 
waveforms are adapted particularly to low signal-to-noise com- 
munication systems. An upper bound for the information rate in 
an additive Gaussian noise channel, based on the use of regular- 
simplex coded waveforms, has been calculated. The results indicate 
that for signal-to-noise energy ratios less than approximately 
unity and for error probabilities ranging from 10 to 10-8, this 
rate is a sizable percentage of the ideal rate which has been derived 
by Shannon and Tuller. 


I. InTRODUCTION 


N his theory for the continuous channel, Shannon has 

| shown that the theoretical maximum for the infor- 
mation rate through a channel may be approached 
arbitrarily closely, if the signals are encoded as noise- 
like waveforms of sufficient length.’’” Rice has developed 
this idea in the case of an additive Gaussian-noise channel 
and has made an analysis of information rates based on 
picking at random the coefficients in a Fourier Series 
representation of the signals*® from a normal universe. His 
work showed that the information rate for such random 
signals approached the ideal or maximum rate only for 
extremely large numbers of symbols, in excess of 2*°, 
and the question arises whether deterministic methods 
of choosing signals might not be better in this respect. 
A deterministic method of picking waveforms is dis- 
cussed in this paper. Specifically, the sample values of 
M waveforms, which represent WM equally probable mes- 
sage symbols, are taken to be the coordinates of the 
vertices of a regular simplex in N dimensions.* ° A simplex 
in N dimensions is bounded by N + 1 intersecting hyper- 
planes; it is the analog of a triangle, which may be 
put in a space of two or more dimensions, and a tetra- 
hedron, which may be put in a space of three or more 
dimensions. If the edges are all of equal length, the 
simplex is regular. The condition that the message symbols 
be equally probable makes it reasonable to assign the 


* Received by the PGIT, March 23, 1959. 

t Res. Lab., General Electric Co., Schenectady, N. Y. 

1C. E. Shannon, “A mathematical theory of communication,” 
Bell Sys. Tech. J., vol. 27, pp. 639-645; October, 1948. 

2C. Ei. Shannon, “Communication in the presence of noise,”’ 
Proc. IRE, vol. 37, pp. 10-12; January, 1949. 

3§. O. Rice, “Communication in the presence of noise—prob- 
ability of error for two encoding schemes,’ Bell Sys. Tech. J., vol. 
29, pp. 60-93; January, 1950. 

4p. H. Schoute, ““Mehridimenzionale Geometre II. Teil, Die 
Polytope,’’ Goschensche Verlagshandlung, Leipzig, Germany; 1905. 

> H.S. M. Coxeter, “Regular Polytopes,’’? Methuen & Co., Ltd., 
London, England; 1948. 

6D. M. Y. Sommerville, “An Introduction to the Geometry of 
N Dimensions,’”’ Dover Publications, Inc., New York, N. Y.; 1958. 


same energy to each waveform; thus the code points | 
equally spaced with respect to each other by virtue 
the geometry of the regular simplex, and the vect 
connecting them to the origin have equal lengths. 

Gilbert has discussed the simplex code in a study 
the relative efficiency of codes for signaling telephc 
numbers.’ This type of code has been considered 
Poritsky in a sphere packing problem.” Similar ideas 
coding have also been employed by Kharkevitch.’ Bas 
has shown that the arrangement of code points afforc 
by the regular simplex corresponds to a minimum of 1 
probability of error when the message symbols are equa 
probable and that the transitional probabilities decre: 
with increasing distance between code points.”? 7 
present discussion is concerned principally with 1 
information rate for the regular-simplex code. Most 
the mathematical details are omitted, inasmuch as tk 
are included in a report by the author, which is genera 
available." 


Il. Tur Vector REPRESENTATIONS 
OF THE WAVEFORMS 


The M waveforms, which represent the message sy 
bols, may in turn be represented by vectors in an 
dimensional space having projections on the coordin 
axes which are the sample values of the waveforn 
If v; denotes the 7th vector representing the 7th wavefor 
then the M/-member column matrix of signal vect 


V; 
eS yy 
LUmu_t 
may be expressed as 
V = XL, 


7H. N. Gilbert, “A comparison of signalling alphabets,”’ 
Sys. Tech. J., vol. 31, pp. 504-522; May, 1952. 

SH. Poritsky, “The Distribution of Points on a Sphere 
Its Application to a Problem in Communication Theory,’’ Gen 
Hlectric Res. Lab., Schenectady, N. Y., Rept. No. 57-RL-1’ 
August, 1957. 

* A. A. Kharkevitch, “To the theory of perfect receiver,’’ Ele 
sviaz, vol. 10, pp. 28-34, 1956; Translation No. R-1956, U.S. Di 
of Commerce, Office of Technical Services, Washington, D. 

0B. L. Basore, “Regular Simplex Coding,’ The Dikew 
Corp., Scientific Rept. No. 1, Contract No. AF 19(604-40 
AFCRC TN-59-165, ASTIA Doc. No. AD-213609. 

uC, A. Stutt, “Regular Polyhedron Codes,’’ General Ele¢ 
Res. Lab., Schenectady, N. Y., Rept. No. 59-RL-2202; March, 1] 


I6O 
here X is an M by N matrix of projections, 


X= 3) 


LV ni X yn_ 


1d L is an N-member column matrix of unit coordinate 
actors. 


(4) 
ly 

, will be convenient to normalize the v, to unit length, 
g., to inscribe the regular simplex in the unit N-sphere. 
Certain pertinent properties of the regular simplex are 
iapted to this discussion of the v; in the following 
atements: 


1) The sum of the & signal vectors is zero, 
| 


M 
| 2% = 0; 
‘ i=1 


consequently, any column sum of the projections 
BE in (8) is zero; 


M 
2 Lin = 0, 
a=1 


2) The distance d,;; between the 7th and jth code 


(5) 


Dim Sle 2a Ns (6) 


points is 
/ 
2M 
C= Ui Vie: (7) 
and the dot product of the two associated vectors is 
1 
: ORS 7 aa (8) 


Because of (8) the correlation matrix, which is the 
matrix of dot products, is particularly simple, for 
all off-diagonal terms are just —1/M — 1: 


ee eal 


VV’ 


I 


(9) 


(-a744) si : J 


where V’ denotes the transpose of V. 

3) A space of at least IZ — 1 dimensions is required 
for the M-Simplex. Furthermore, the spacing of 
code points cannot be increased by adding dimen- 
sions under the constraint of fixed length vectors, 
1.e., fixed energy waveforms. 

4) The regular simplex method of locating M points 
on the unit N-sphere is optimum in the sense that 
no other method of location can make any of the 
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spacings of code points greater then that afforded 
by such equi-spacing without decreasing the spacings 
of other points. 


An iterative procedure for constructing a simplex in 
N dimensions is to build onto a simplex in N — 1 dimen- 
sions by joing the N vertices of the (NV — 1)-simplex 
to a point outside this space.*’” A particularly advanta- 
geous orientation for purposes of analysis of the regular 
simplex code is obtained if this point outside the space 
is taken to be the point ene on the new coordinate axis. 
To construct a regular simplex in N dimensions 1) the 
(N — 1)-simplex must be regular; 2) it must be translated 
upon addition of the new point so that (6) is satisfied; 
and 3) it must be scaled so that it is inscribed in the 
unit N-sphere. To illustrate this procedure in the case 
N = 38, one starts with vectors associated with a regular 
2-simplex (equilateral triangle) oriented so as to give 


V2 = — v3 The: us 1 
LZ ens ») 1 D 2) 
V3 1 
Vo = 9 1k 9 ile (10) 
032 On el:. 


where the second subscript denotes the dimensions of 
the space. The vectors associated with a regular 3-simplex 
(tetrahedron) include the unit vector in the third dimen- 
sion and scaled and translated versions of the vectors 
from the 2-space: 


1 Says 1 V8 1 

if Rea he ANE Ee 3 le — 3 ls, 
1 V3 V8 aye 1 

Yas = Astor — 3 le = ere 3 lz — 3 ls, 
1 8 1 

V33 = A3V32 — 3 1,=0O1,+ ve 1, 3 ls, 

Mw =~ 01,4 01,4 13, (11) 


where the scaling factor A, must be +/8/3 to make the 
new vectors unit length. 
If this procedure is continued to more dimensions, one 


sees that the pth column of the X-matrix of projections, 
pi Ved. ott Me. Lis: simply 
Ke 2 
Lip | eos F(M, Pp) 
Pp 
tm | | —> FM, p) 
aa (12) 
12 
vp+1,p FCM, P) 
Un+2,0 0 
mp _| ls 0 2! 
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where 


il 1/2 
M1 
1+ 

D 


an 


F(M, p) = (13) 


The vectors determined by the projections in (12) 
will be referred to as prototype vectors and are expressed 
in terms of the minimum number of dimensions. If NV 
in (8) and (4) is greater than AJ — 1, then there are 
superfluous dimensions and the last VN — (MM — 1) columns 
of (3) are null. 

Insofar as calculations based on the central values of 
the correlation functions, which are given in (9), are 
concerned, these prototype vectors are very useful; how- 
ever, the corresponding waveforms are not noise-like, 
and the fact that the off central values of the correlaticn 
functions are not small would produce deleterious effects 
in a communication system wherein timing is not known. 
For example, the last two vectors are 


l ag ge 


Uuet = 


re ae 
<S M-1) 
Wit (14) 


Um = Iw-1, 


where it is seen that a delay of the (WZ — 1)st waveform 
by one sample interval giving 


VUur-i (shifted) 
1 a 1 
ge Te Feo igh ecm ae 55 


changes the cross correlation to [1/(IZ—1)] VW M(M — 2), 
which for large M is only slightly less than the central 
value of unity for the auto-correlation functions. Thus, 
in the presence of noise, such cross-correlation peaks 
could not be distinguishable from auto-correlation peaks, 
and false or ambiguous detections would result. 

In order to improve the waveforms in this respect, 
one may generate a new set of vectors by rotating the 
prototype vectors in the space, or, equivalently, project 
them onto a rotated set of coordinates. Thus, waveform 
design may be regarded geometrically as a problem of 
rotation. The number of used dimensions may also be 
increased any desired amount in the process, which would 
be advantageous if the noise is power limited rather than 
power-density limited. This rotation implies finding an 
orthogonal matrix A such that 


L= AK (16) 


where K is a column matrix of k,, p = 1, 2, +>. SN, 
the unit coordinate vectors in the new coordinate system. 
Thus, 


(15) 


V = XL = XAK = YK (17) 


where Y = XA may be identified as an M by N matrix 
of projections in the new coordinate system. Actually, 
since the last N — (M — 1) columns of X are null, only 
M — 1 rows are required in the A matrix. 
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It is beyond the scope of this paper to discuss 
picking of an A matrix which will rotate the coordin 
system so that more suitable correlation functions 
obtained. However, if N is large, a random rotation n 
be satisfactory for some purposes; in which case an 
matrix which is orthogonal only in a probabilistic se 
may be formed by picking its members from a nor! 
universe. 


III. InrorMATION RATE 


A communication system which might employ regul 
simplex coded waveforms is shown in Fig. 1. The wa 
form generator may be thought of as a tapped de 
line filter, for purposes of illustration, even though st 
a generation technique may not be practical when 
is very large. A particular waveform is selected for e 
transmission and the received waveform is this wavefo 
perturbed by the additive channel noise. The recer 
includes a set of WM filters which are individually matek 
to the M possible transmitted waveforms. These matel 
filters perform the operations of cross correlation wh 
are ideal according to the theory of Woodward a 
Davies.’” The outputs of these filters are fed to a decis: 
device which selects that output which is both the larg 
of all outputs and exceeds some threshold level. 


Waveform Waveform Decision 


Generator 


Selector Circuits 


Noise 


Message Symbol acat 


Input 


Matched Filters 
{Output is dot product 


o*% 


(vj +n) ) 

Fig. 1—Matched filter communication system employing reg 
simplex coded waveforms. The 0; --- vy are the vector r 
sentations of the waveforms. 


In order to determine the maximum information 
for this code, attention will be focused on the cen 
values of the correlation functions. Thus, if y = 2; 
is the received vector when v; is transmitted and n 
perturbing noise vector, the output of the jth filte 
the quantity C; which is 


| 
| 


C; =9;-@; +7). 


For a correct detection when v, is sent, then C, 
satisfy 
C.> C; 


j=1,2,-++,¢-1Li+1,-,M 


_ P.M. Woodward and I. L. Davies, “Information theor: 
inverse probability in telecommunications,” Proc. IEE, vol 
pt. 3, pp. 37-48; March, 1952. 


a0) 
d 
C; > Cr 


ere C’, is the threshold level which is set to reduce the 
mber of false indications when no message is sent 
when the noncentral values of the correlation functions 
> being computed when a message has been sent. It 
wssumed that these noncentral values have been suitably 
atrolled in the waveform design so that the number 
false indications arising from them is negligible. 

The message symbols being equally probable, the 
priort probability for the occurrence of a particular 
sssage vector v; is 1/M, so that the rate information 
fed into the channel is 


H(v) = log,M bits/symbol. (19) 


ter a decision in favor of v; is made at the receiver, 
> situation is described by the probability diagram of 
x. 2. Here the probability thatv; was sent is 


sere Pp is the probability of detection and Pz, is the 
obability of error. By virtue of the symmetry of the 
sular simplex, the probability that v;, 7 ~ 7, was trans- 
tted when v, is received is Pz/(M — 1). The equivoca- 
m as computed from Fig. 2 is, therefore 


@ = —(1 — Pz) log, (1 — Pz) 

: og its eymbol: 61) 
i D 1082 pan 1ts/Sym i 

| All Message 

_ Vectors ; 

' Possible at 5 v. (Indicated at 


Transmitter Receiver) 


Path weights denote probability 
that, when v, 1s indicated at 


receiver, it was produced by vj 
at the transmitter 


. 2—Probability diagram for regular simplex code when 0; is 
indicated at receiver. 


+ is assumed that the waveforms are sent sequentially, 
1 that no ‘‘dead’’ time exists between waveforms. Thus 
s number of waveforms per unit time is just 1/7 = 
"/N, where T is the duration of the waveforms and W 
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is their bandwidth. The actual information rate is now 
Rk, = ; [H(v) — H,(v)|_ bits/second, (22) 


and after some mathematical manipulation,” a first order 
approximation for the case cf large N is 


R, & W log, {1 ate - Zn 
oan cea 


Channel Input Terms 


ey Pe \ bite/second, (23) 


— W log, {1 4 


Equivocation Terms 


where LH, is the signal energy, H, the noise energy in a 
period 7 and bandwidth W, and p” = N(E,/E,)™ 

~In general P, will be small so that the equivocation 
term in (23) may frequently be neglected without serious 
error, so that 


RW loge {1 4 een 
Ee 


\ bits/second. (24) 


If this expressicn is compared to the ideal information 
rate for a channel with additive white Gaussian noise 
as given by Shannon and Tuller,’’”’”’ 

E, : 

R; = W log. {1 + EA bits/second, (25) 
it is seen that the effective signal-to-noise ratio is reduced 
from its actual value by the factor (2 ln M)/p’. It remains 
to determine the relationship between M and p° for 
prescribed P, so that the relative values of R, and R; 
may be evaluated. 

As was stated above, only the central values of the 
correlation functions will be considered; and it is, therefore, 
permissible to use the prototype vectors in the ¢alcula- 
tions, even though rotated vectors may have been used 
in the actual transmission. One might imagine that these 
transmitted vectors are given a rotation at the receiver 
to return them to the orientation of the prototype set. 
The statistics of the noise are not changed by such a 
rotation, and the noise vector projections (or sample 
values of the noise) are Gaussian before and after this 
transformation. It will be a further convenience to assume 
that the actual transmitted vector is rotated into the 
position of vy, [see (14)] so that it is just the unit vector 
ly_,. Alternatively, one might evaluate Pp and P, for 
that fraction 1/M of the times in which vy, is transmitted; 
then by symmetry, the same values must hold for any 
other member of the code. 


13 Stutt, op. cit., section V. : 

“4 The quantity p? is identical with the fundamental quantity 
R introduced by P. M. Woodward, “Probability and Information 
Theory,’’ McGraw Hill Book Co., Inc. New York, N. Y., p. 87; 1953. 

165 W. G. Tuller, ‘Theoretical limitations on rate of information 
transmission,’ Proc. IRE, vol. 37, p. 468; May, 1949. 
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If p(Cu, Cu-1, --: , Ci) denotes the jomt density 
function for the matched-filter outputs, then 


coe) Cm 
Pp = dC 4 i dC y—1 
Cr —c 


Cm ; 
sf dCipCu, Cure). 26) 
The joint density function for the Cy_; may be written 
in terms of the conditional density functions, 7.e., 


PCr, Cue; ipa C)) 


= p(Cu)p(C u-1 | Cy) v8 pC, | Car Cope es Cz; 


(27) 


which can be written explicitly by virtue of the simple 
expressions for the projections, (12). Because of (5), it 
follows that 


M— 


eel ? 


so that the density function for C,, all other Cy_; known, 
is a delta function. The remaining M — 1 conditional 
density functions are Gaussian in every case,'® and (26) 
may be rewritten as : 


a5 / exp {8 en Wente ) *} aC 
au JCr 


p M-—-1 
ot WM(M — 2) 


(28) 


eee _# wv gta) 
ie exp { 2 MM —a_\or+r7 IC ur 
pp (w= DpM—h 
A/ Qr Mi Corae 1) 
iF o’ (M — 1)(M — k) 
[oot Fire 
(Cx. ke + 7 aay 2 Ce A) baw 


er eches p 2M — 1) 
~ * exp {- Die te Ne 


pos =) 
lA Dre M 


i M-3 2 
{c. as ) y Ci) aes 


M—2 


CM 
f i(c, =} > opal dC. 


p=0 


(29) 


The integral involving C,, denoted J,, may be either 
1 or 0, but has the expected value 


(30) 


Negligible error will result for the values of Pg and M 
of interest, if the value of J, is taken as 1. 


16 Stutt, op. cit., Appendix A. 
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If in the general integral J, involving Cy-;, a chan, 
of variable is made, 


tae eG — 1)(M — k) 
us = 75 M[M — (k + 1)] 
(Caos + aq & Cu), (3 
Wes O ieee teem 


then (29) may be rewritten as 


co 


1 -2?M 
P me | eo AZ, 
2 Va J-0/20-¢r) J 
le pee dZ 
A! x M-1 
1 UM—-k ae? 
6 eeu i é AG OLn =e 
Tv 
1 ie 5 
al e-7"* dZa, (3 
T J—2 
where 
eee p (M 1)( k) 


M[(M =(k+ DY] 
1 k-1 
it + j) ay 2d ea) 


k="1, ,M — 2. (3 


Ve 


Because the Uy,;_;, are random upper limits and functio 
of all the variables to the left of the integral J,, it h 
not been possible to carry out the multiple integratio 
(32), exactly. An approximate solution for Pp is offer 
here which involves using the expected value of t 
Uy-, as an upper limit in the integral 7;,, which is 


M(M — k) 
(M — 1)[M — (k + 0] 


U —k oT, oe 
( M ) 4/2 


a te ,M — 2. 6 


By using (Uy_;) instead of Uy_;, the multiple integi 
is replaced by a product of integrals, each of which m: 
be evaluated in terms of the error function. The resulti 
approximation to Pp is proposed as an upper bound of F 
and this bound should become very close to the exa 
solution as p increases: Note that (Uy;-,) is positiy 
therefore, a fluctuation of Uy, below (Uy-,) will ¢ 
crease the value of J, more than an equal fluctuation 
Uy-, above (Uy-,) will increase it. Furthermore, t 
variance of Uy,_; is 


— (k — 1) 
eae 


which is independent of p; consequently, the larger t 
value of p, the further out on the tail of e~7’™-* t, 
integral is taken, and the effect of fluctuations of U 
on the value of J, becomes less and less. 


(U4 M- A= (Uy Ae: G: 


960 


The value of 7, with (Uu_,) used as the upper limit 
denoted J,, and has the value 


1 (UM —k) - 
= = i @ 4 boas Wi ie 
M(M — k) 


ee Vai — DM woes) ee 


here E(_) is the error function. The approximate expres- 
on for Pp may now be written as 


.~ pr os x) wa (hee en |} 


lt + (5 ort 


— 1I)[M —-(k+ =} 
or a reliable detection system, p will be moderately 
irge (perhaps in the vicinity of 10 at least) so that the 
rguments of the error functions are large permitting 
1e use of the asymptotic expansion for H( ). This gives 


(35) 


M—2 


P(D) & Do eh (36) 


1l—er — 


here e,7 is an error due to missed detections and is given 
Wy. 


— 


: exp{ 2 = ay ; a 
= ie ; (37) 

Va Pa (1 > Cy) at Dae ”) 

: V2 

| 

nd pe? «, is an error due to erroneous detections 

fere €, 1S given by 


| o M(M — *) 
a i — iM = (he 1) 
2/n  p M(M —k) 
| \/2 VM — DIM — (k + 1) 
(M — 1)[M 


{1 - =k nih 
| p M(M — k) ; 


Tf the threshold is set so that 7 is small compared to 
zy, a . . . 
ty &, an approximate expression for Py is 


M-2 
Pe. ee. 
k=1 


(38) 


(39) 
| 
fter some mathematical manipulation,’’ an approximate 
spression for P, valid for values of M in the vicinity 
* 10 and larger is 


eee 
© p.A/ 2x M 


oo -£ (72a) 


17 Tbid., Appendix B. 
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and when MM is very large this expression simplifies to 


M il ‘ 
Py ce 1) ; St M 
BR / oe 2 exp 5 (M large) 


A straightforward calculation for the cases M = 2 
and M = 3,” yields the following expression for Pz: 


(41) 


2 


J pred M = 2,3. (42) 


p V2 (1- 4 


IPs BS 


In arriving at (42), only approximations for the error 
function for large arguments were required, and the 
“mean upper limit”? approximation used for large values 
of M was not used. 

Eqs. (40), (41), and (42) have been used to calculate 
the curves of Pz as a function of p’ for fixed values of 7 
shown in Fig. 3. These same results are shown in Fig. 4 
with M as a function of p’ and P, the parameter. The 
curves of Fig. 4 provide the relations between M and 
p for fixed P, which are needed to complete the calcu- 
lation of R,, the actual information rate in (23). Ac- 
cordingly, ratios of R, to the ideal rate R;, (25), have 
been calculated for values of Py, ranging from 107” 
10° and are plotted against log,) M and M in Figs. 5 and 6. 

It will be noted from statement 3 in Section II that the 
maximum value of JM for a simplex code is 


Max = N +1; (43) 
hence from the definition of p’, 
Ei 
ee 2c Z 
Meas mae ie, == le (44) 


The values cf W,,,. for the specified values of imput 
signal-to-noise ratio H,/H, are indicated by the dashed 
curves in Figs. 4, 5, and 6. The intersection of these 
dashed curves with the curves of M/ vs p’ in Fig. 4, then, 
are the limits imposed by the simplex geometry on the 
values of M which may be used with a given input signal- 
to-noise ratio and a given probability of error. 

These limiting values of 1/7, as shown in the rate curves 
of Figs. 5 and 6, illustrate the applicability of simplex 
coding to low signal-to-noise ratio systems. To the right 
of the dashed curves, more code points should be dis- 
tributed over the N-sphere than is made possible by the 
simplex. The curves of Fig. 5 are for values of H./E, 
less than about 0.1. Here the dependence of R./R; on 
E,/E,, is negligible, so that one curve suffices for each 


18 Tbid., Appendix C. 
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aeeer 2 
t OF © ce} oe 
: 
, | 18 
et 
ee alee 
aa ES at 
| 
beset [ 
ft 
r 1 { 
ia 
oy stad il = alo=-I="se /E =3 
[ q "a = — - =o heel iiallie tio i 
ZG eet SIE,/E,, 210 
PAG hE Si 
z os sete tls sel = 
2 iz 
) 10 20 30 40 50 60 


Fig. 4—Relationship between number of message symbols M and 
p” for fixed values of Py. Dashed curves give maximum values 
of M, which are possible with regular simplex code, for the in- 
dicated input signal-to-noise ratios. 


value of Pz. The curves of Fig. 6 are for values of /,/H, 
in the vicinity of 1. At these higher signal-to-noise ratios, 
R,/R; improves noticeably with #,/H,,, hence each curve 
is identified by the value of both H,/H, and P,. Note 
that except for extremely small values of Pz, the equi- 
spacing of code points afforded by the simplex should 
not be used for signal-to-noise ratios much above 10. 

It would be instructive to compare the information 
rate curves with those obtained by Rice.’ Such a com- 
parison is not readily made inasmuch as Rice’s curves 
apply for values of M@ much in excess of 2*°. However, 
his curve for the case of Pz = 10° and E,/E, = 10 
has been extrapolated to values of JZ under 10° and is 
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Fig. 5—Ratio of actual information rate R4 to ideal rate for addit 
white Gaussian noise channel #; for regular simplex code ¢ 
for low input signal-to-noise ratios (<0.1). 
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Fig. 6—Ratio of actual information rate (R.) to ideal for addi 
white Gaussian noise channel (R,) for simplex code and 
input signal-to-noise ratios in the vicinity of unity. 


shown in Fig 6. Such a limited comparison indica 
that for the same conditions, the deterministic simp 
code does give a more rapid approach to the ideal 
formation rate than does a random selection. 


IV. ConcLusions 


The regular simplex affords a convenient approach 
equi-spacing code points in an M-symbol continuous co 
Such a code is particularly applicable to low signal- 
noise ratio systems, and the actual rate of transmiss 
of information in an additive white Gaussian noise chan 
is a large fraction of the ideal rate given by Sham 
and Tuller. 

Because the simplex method of coding is readily ada 
able to any number of symbols, from two upwards, ¢ 
to an arbitrary number of dimensions (V > M — 
it may be of considerable practical interest. While 
simplex fixes the relative position or spacing of the ¢ 
points and the central values of the correlation functi 
the actual waveforms are arbitrary at this point. A me 
problem remaining with this method of coding, then 
that of waveform design, and this, as has been seen 
a matter of suitably orienting the simplex in the N-sp: 


I6O 
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sonstruction of Relatively Maximal, Systematic Codes of 
Specified Minimum Distance From Linear Recurring 
Sequences of Maximal Period* 


C. N. CAMPOPIANO}, MEMBER, IRE 


Summary—Relative to a distance function which is both trans- 
tion-invariant and expressible as the sum of the distances between 
ordinates, an upper bound is obtained for the size of certain 
, d) systematic codes. This bound is closely related to a result 
'M. Plotkin. It is shown that certain (n, d), systematic codes 
stainable from linear recurring sequences are of maximal size 
an appropriate class of systematic (n, d) codes when the distance 
action is translation-invariant and the sum of the corresponding 
ordinate distances. The results are specialized to the Hamming 
stance and to the cyclic distance of C. Y. Lee. Relative to the 
amming distance, the results are valid for an arbitrary Galois 
sid GF(q). For the cyclic distance, however, the results are valid 
ily for prime Galois fields and for GF(4). Moreover, it is shown 
at for the latter distance, it is impossible to set up a ‘‘translation- 
variant, coordinate-sum”’ distance which is also cyclic for any 
mprime Galois field except GF(4). 


I. INTRODUCTION 


and notation, and indicate the organization of the 

Femainider of the paper. Let GF(q) be a Galois field 
"gq = p' elements where p is a prime and k is a positive 
peer Let V,(qg) be the set of all g” ordered n-tuples 
= (6,, -~- , 8.) = (8:1, with coordinates in GF(q). 
a Unless Relreawine stated, elements of GF(q) will be 
snoted by Greek letters, while elements of V,(q) will be 
ssignated by ordinary lower-case: a, b, c, --: . If a is 
_V,(q) its coordinates will be represented by the cor- 
sponding Greek letters with appropriate subscripts; thus 
= (a, °°: , @,) = (a;)7-1. Any subset of V,(q) is called 
block code,’ or simply a code, cf length n. Members of 
n(q) are variously called words, vectors, points or sequences. 
fembers of a block code are called code words, code points, 
e. The size of a block code C, denoted by “| C |”, is 
ie number of code words in the code. Relative to the 
sual operations, 


1 


1, eee 


[ N this section we introduce basic concepts, terminology 


? Qn) 


7 An a Gi), 


) Qn) , (a1, Og, °°* 


(a, + fi, cae 


itn) = (aa,, °°" 
+ (61, °°° , 8) = 
(a, as, Bs, GF (q)). (1) 


.(g) is an n-dimensional vector space over GF(q). [In 
); “(a4; ae Qn) a (6:1, Site G Oenle means “a; = 6; 
m= 1, 2, n.’’| We write 0 = (0, 0, , 0) for 
e additive identity element of V,(q). A subset of V,(q) 
| 

cag d by the PGIT, August 14, 1959. 

bone B atenis Div., Radio Corp. of America, Camden 2, 


iz. P. Lloyd, “Binary block coding,” Bell Sys. Tech. J., vol. 
, Pp. 517-535; "March, 1957. 


is called a systematic code if it is a linear subspace of 
V,(q). ° A code in V,(q) is called nontrivial provided 
that for each [2] between 1 and n there is some code word, 
say (%1, -*: , %,) such that xz, ¥ 0; all other codes are 
called trzv¢al codes. All codes discussed below are supposed 
to be nontrivial. 

Suppose that a distance function, d(a, 6), is defined 
on all pairs a, b of V,(g). A code C in V,(g) for which 
d(a, b) > d for all elements a and 6 of C is said to be an 
(n, d) code. If the size of the (n, d) code C is not exceeded 
by the size of any other (n, d) code, then C is said to be 
maximal. We are really interested, in the following develop- 
ments, in a kind of relative maximality as follows. A 
systematic (n, d) code whose size is not exceeded by that 
of any other systematic (n, d) code is said to be relatively 
maximal. 

In Section II, we present some general results on 
systematic codes some of which are valid only for a 
distance function of a special type (which we have called 
a translation-invariant, coordinate-sum distance). Section III 
summarizes relevant results from the theory of linear 
recurring sequences of maximal period.*” From these 
sequences we construct, in Section III-A, codes of known 
minimum distance which are relatively maximal relative 
to a translation-invariant, coordinate-sum distance. In 
Section IV, these results are specialized to the Hamming 
distance.”’* For prime Galois fields, the results of Sections 
II, III, and III-A are applied to Lee’s cyclic distance” 
in Section V. In Section VI, the results of Section V 
are extended to GF(2’). It is also demonstrated there 
that it is impossible to define a Lee cyclic distance which 
is a translation-invariant, coordinate-sum distance for 
any nonprime Galois field other than GF(4). Finally, 


2R. W. Hamming, “Error detecting and error correcting codes,’’ 
Bell Sys. Tech. J ” vol. 29, pp. 147-160; April, 1950. 

3D. Slepian, “A class of binary signaling alphabets,’”’ Bell Sys. 
Tech. J.. vol 35, pp. 203-234; January, 1956. 

«B. M. Dwork and R. M. Heller, “Results of a geometric 
approach to the theory and construction of nonbinary, multiple 
error and failure correcting codes,’’ 1959 IRE Natrona Con- 
VENTION pee pt. 4, pp. 123-129. 

5 N. Zierler, “Linear recurring sequences,” J. Ind. and Appl. 
Math., vol. 7, pp. 31-48; March, 1959. 

6S. W. Golomb, “Sequences with Randomness Properties,”’ 
The Glenn L. Martin Co. ., Baltimore, Md., Terminal Progress 
Rept.; June 14, 1955. 

TODS: he Huffman, “The synthesis of linear sequential coding 
networks,” Third London Symp. on Information Theory, London, 
Eng., Butterworths Scientific Publications, London, Eng., Colin 
Cherry, Ed., pp. 77-95; 1956. 

8'C. Y. Lee, “Some properties of nonbinary error-correcting 
codes,” IRE Trans. on INFoRMATION THEORY, vol. IT-4, pp. 77-82; 
June, 1958. 
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Section VII contains a single example of the preceding 
analyses for the special case GF'(4). 

Before proceeding to the developments outlined above, 
we add a few remarks concerning the definition of the 
term systematic code given above. We could have genera- 
lized the concept of systematic code given by Hamming” 
and Slepian’® as follows: 


Definition 2: A systematic code in V,(q) is a set of 
elements a = (a;)j, b = (@,)1, «+: in V,(g) determined 
by the following rule. For a certain fixed set of k indices, 
say 71, °°: , 7, the elements a;,, -*+ , a;,, called ¢nforma- 
tion digits, are assigned arbitrary values from GF‘(q), 
while each of the remaining n — k elements, say 
ines) °° * ) Qin, Called check digits, is a fixed linear combi- 
nation of the information digits, say 


k 
an = Doane Me{ te +1, Gaar Slats 


7=1 
The k-tuple (2;,, -- 
in V,(q). 

But then such a code is necessarily a linear subspace 
of V,(q). Moreover, one can readily modify Slepian’s 
proof’ of his Theorem 3 to show that every linear subspace 
of V,(q) is a systematic code in V,(q) according to defini- 
tion 2. It is an interesting fact that every additive subgroup 
of V,(q) is a linear subspace of V,(q) if and only if q 
is a prime. This explains the difference between Slepian’s 
terminology in his Theorem 3 and our terminology above 
which stresses linear subspaces. On the other hand, 
generalization of Slepian’s decoding scheme® to certain 
symmetric, nonbinary, discrete channels without memory 
utilizes the group cencept alone. 


- , @,) assumes each of the g” values 


II. GENERAL RESULTS FOR SYSTEMATIC CopES 


In the present section, we generalize a result of 
Slepian’s.” This result is applied to the case in which a 
special kind of distance function is defined on V,(q) 
to obtain a partial generalization of a result of Plotkin’? 
on the maximal size of systematic (n, d) codes. For non- 
trivial codes we have the following result. 


Theorem 1: Let C be a nontrivial code in V,(q) of 
size |C | = g™ (1 < m < n). Let the code words of C 
be (eh ** 1) Cale = pepe ge herein theaset 
(Crzy Caiye Cais, 2h Clty} G1, 2y* +s 2) -each’of the 


q elements of GF(q) occurs precisely g”~* = (1/q) | C | 
times. Hence, in the code C each of the q elements of 
GF (q) occurs precisely (n/q)'| C | times. 


Proof: For each ~ in GF(q), let N; be the number 
of code words in C whose first coordinate is €. In particular, 
No is the number of code words in C whose first coordinate 
is 0, where 0 is the additive identity of GF(q). We shall 
show that NV; is the same for all £, hence has the value 


9D. Slepian, op. cit., p. 228. 

10 M. Plotkin, “Binary Codes with Specified Minimum Distance,” 
M. S. thesis, Moore School of Elec. Engrg., University of Pennsyl- 
vania, Philadelphia, Pa.; June, 1952. 
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g”" *. Let a be an arbitrary nonzero element of GF 
There exists some code word, say ¢ = (71, Yo. °°" 

such that y, + 0. Hence there exists at least one cc 
word with a as its first coordinate, namely ay;*c. That 
N, > 0: Let a; @ = 1, 2, --* , Nz) be the Vi, dise 
code words with a as first coordinate. Then a, — 

Mz — Q, M3 — ,°** , Gy — a, are N, distinct code wo. 
with 0 as first coordinate. Hence No = N,.. Let . 
Zy, ‘+: , Zy, be the distinct code words with 0 as fi 
coordinate. Then, since a, has @ as first coordina 
Z; +a, 7 = 1, 2, +--+ , No are distinct code words w 
a as first coordinate. Hence N, > No and Ny = I 
But a was an arbitrary nonzero element of GF(q). He 
for all £ ¥ 0, Ne = No = gq” . The remainder of 7 
proof is clear. 


Corollary 1: Let F(a) be a real-valued function defi 
on V,(q) by the relation 
F(a) = DX fla.) = (a1, ae yea); 
where f(a) is a real-valued function defined on GF 
Let C be a nontrivial code in V,(q). Then 
n 
DL F@)= ||. 2a ier 
aeC q aeGF(q) 

The above results are particularly significant in sit 
tions where we have a distance function of a spe 
type which we have called a translation-invariant, 
ordinate-sum distance. (We are not pleased with t 
terminology, but some simple labels seemed appropri 
in order to underline the specialized nature of the 1 
lowing results.) We introduce a translation-invaria 
coordinate-sum distance on V,(q) as follows: Let th 
be defined for each a of GF(g) a norm or weight | a | st 
that | a | is real-valued and 


(m) | —w| =|e|;|0| =9, 
(na ~ 0 ea ee a6 
(ms) |eo+B|<lal+ |B. 


implies 


Then 
(4) | a | =, 


and the function p(a, 8) = | a — B | is a distance funct 
on GF(q). Perhaps we should call | @ | a “weak” no 
since we do not demand that | a8 | = | a | | @ |."* He 
ever, we shall simply say “norm.” We define the nc 
or weight of any element, b = (6, --- , Ba) of Vn(q) 


oil= D1 asl. 


Then || 6 || is real-valued and satisfies (n?), +++ , ( 
when “| |” is replaced by ‘‘{| ||”. We define the dista: 
d(a, b), between two elements a = (ay, «+ , an) 0 
| 


" M. H. A. Newman, “Elements of the Topology of Plain 
of Points,’ Cambridge University Press, Cambridge, Eng.; 1) 


960 


Fi eg Bn) of V,(q) as d(a, b) = | a— b I. Clearly, 


Sla-el @) 


Je call d(a, b) a translation-invariant, coordinate-sum 
istance because of (3) and because d(a, b) is translation- 
variant (d(a + c, b + c) = d(a, b)). Note that p(a, 8) 
also translation-invariant in that 


Ry ere ay = pare): (4) 


In the above, we started with | a |, defined p(a, B) = 
a — 6 | and continued to define || b ||, etc. Alternatively, 
iven a distance function, say p(a, 6B), for GF(q) such 
hat p(@, @) is translation-invariant, we can define | a | = 
fa, 0). Then | @ | satisfies (n?), --- , (v4) so that it is 

genuine norm as defined above. The definitions for 
6 || and d(a, b) can then proceed as above. Alternatively, 
‘re could postulate a translation-invariant distance func- 
ion d(a, b) for V,(q) and a function p(a, 8) such that 

(a, b) = >2%_, p(a;, B,). Then p(a, B) is a translation- 
ivariant distance function for GF(q), and the functions 
{a, 0), p(a, 0) play the roles of the norms || a ||, | a | 
atroduced above. 

Let S,(n, d) represent the size of the largest nontrival, 
ystematic (n, d) code in V,,(q). From the above definitions 
ad Corollary 1 we readily obtain Theorem 2. 


Theorem 2: If 


n 


d(a, b) = D pla, 8) = 


t1=1 


d>* lo |, 
Y aeGF(a) 
hen 
S.(n, d) < —_—* 
se | a | 
qd aeGF(a) 


rhenever the distance function, d(a, b), is a translation- 
avariant, coordinate-sum distance function defined in 
erms of | a | as in (3). 


Proof: Let C be a nontrivial (n, d) code in V,(q). 
‘hen d(a, b) = d for all a, b in C. In particular, d < 
(a, 0) = || a || for all nonzero a in C. Hence, since there 
re (| C | — 1) nonzero terms in C’, we have 


Paine e=eet Os) =) a. 


‘herefore, from Corollary 1, and (2), 


me 


2GF (aq 


Bei PoalieetC. 1) a: 
qd a ) 


‘hen the theorem follows directly. 


III. Lingar RECURRING SEQUENCES 
or MAXIMAL PERIOD 


In this section we utilize the notation and terminol- 
gy of Zierler® in summarizing certain results from the 
heory of linear recurring sequences which are required 
1 the following sections. We consider linear recurring 
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sequences over the Galois field GF(q), ¢ = p*. Recall 
that a linear recurring sequence of mth order is a sequence 
{a;}{ which satisfies a linear recurrence relation, 7.¢., 
a relation of the form 


Yo + Vi@n-1 + °° + YmOlk—-m = 0, 
k=m+i1,m+2,--- (5) 


with a, y,,¢ GF (q) and yo¥m 4 0. The mth order recurring 
sequence is uniquely determined by the relation (5) and 
by the first m terms a1, a2, ++ , &m. It is known that these 
sequences are periodic with periods not exceeding g” — 1; 
that is, there exists a 7 < q” — 1 such that a;.7 = aj, 


a= 1,2,---. There are exactly ¢(q” — 1)/m translation- 
distinct linear recurring sequences of order m with period 
q” — 1 [¢(s) is the number of integers in the set 1, 2, --- , s 


which are relatively prime to s]. Following Zierler we 
call such a sequence with the maximal period an M 
sequence. We shall use M sequences to construct syste- 
matic, relatively maximal (n, d) codes. First of all, how- 
ever, we recall a few facts about M sequences and their 
recurrence relations. 

The characteristic polynomial f(x) of the relation (1) 
is defined as 


f(x) = Vnt == Vo + oe a + Vit = Yo: (6) 


It is known that a necessary (though not sufficient) 
condition that the nonzero sequences determined by (5) 
be M sequences is that the characteristic polynomial (6) 
be irreducible in the ring, GF(q)[2], of polynomials with 
coefficients in GF(q). Some irreducible polynomials are 
given for GF(p), p = 2, 3, 5 and 7 by Church.” A few 
polynomials which are irreducible over GF(2’) are dis- 
played in Section VII of this paper. 

Let f(z) be an irreducible polynomial which is the 
characteristic polynomial of an mth order linear recur- 
rence (5) which yields M sequences. Let G(f) denote the 
set of all sequences generated by (5). There are g” elements 
in G(f) including the zero sequence, ¢ = (0, 0, -::). 
All gq” — 1 nonzero sequences of G(f) are M sequences 
of period g” — 1. The M sequences of G(f) differ from each 
other merely by a translation; 7.e., if {a;}9, {6:}4 © G(/) 
and are nonzero, then there exists an integer s such that 
Bi+, = a;,¢ = 1, 2,3, --- . If {a;}4 is an M sequence 
of G(f), then every subsequence {a,}°**"*"-* has the 
property that each of the (g” — 1) ordered in m tuples 
(B,, -*: , Bn) with elements in GF(q) (exclusive of the 
all-zero m-tuple) occurs in the subsequence precisely once. 
It is known that when G(f) contains M sequences then 
G(f) is an m-dimensional vector space relative to the 
operations 


(7) 
2 R. Church, “Tables of irreducible polynomials for the first 


four prime moduli,’’ Annals of Math., vol. 36, pp. 198-209; January, 
1935. 
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for {a,}%, {8:}4 e Gf), a e GF(q). The preceding well- 
known results yield directly the following. 


Theorem 3: Let {a;}7 be an mth order M-sequence 
(with period g” — 1) over the field GF(q). Define the 
set of (q” — 1)-tuples dp, a, +++ , Aqm—1 a8 
dy) = (0,0, --- , 0) (8) 


, m 
Q; = (@:, O41) °°° QpegiN 2), c= ines eee jG — j. 


Then for 7 07 ¥ 0, a; and a; are cyclic permutations of 
each other. If a; is in the set a, +++ , @gn_1, then 0 occurs 
exactly q”~’ — 1 times in a, while each of the (¢ — 1) 
nonzero elements of GF(q) occurs precisely gy” * times 
in a;. The set (8) is an m-dimensional linear subspace 
of Van-1(q). 


A. Application to the Construction of (n, d) Codes 


Let a translation-invariant, coordinate-sum distance 
d(a, b) be defined for V,(q) in terms of the norms | a |, 
|| a || and the distance function p(a@, 8) as indicated in 
Section II. The results of Sections II and III then yield 
Theorem 4. 


Theorem 4: The set (8) is a systematic code of size 
q” in V,(q) (n = gq” — 1). Relative to a translation- 
invariant, coordinate-sum distance, the distance between 
cede words is d = g” ‘N, where 


Ds 


aeGF (aq) 


N = 


| o |. 


The code (8) is a relatively maximal (q” — 1, q”'N) 
code. In fact, 


Sha —"1,.¢° 9) = 9". 


Proof: For a; # a; in (8), d(a,, a;) = || a; — a; || = 
[| a, || = @” °N = d. Moreover, 


nr er aa), 
qd q 


Hence, from Theorem 2 


But the size of code (8) is g”. Hence S,(n, d) = q” and (8) 
is indeed relatively maximal. 


TV. APPLICATION TO THE HAMMING DISTANCE 


We utilize the definition of the (generalized) Hamming 
distance as given in Dwork and Heller,* but we introduce 
it in terms of the concepts used in Section II. Thus, we 
define the Hamming norm of elements, a, of GF(q) as 
| a | = 1 or O according as a@ is nonzero or zero. The 
Hamming norm of any b = (6, :-+ , B,) in V,(q), namely 
|| ® ||, is defined as in Section II, #.e., || 6 || = Dot. | 8; |. 
Finally the Hamming distance, d(a, b), between any 
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elements a, b in V,,(q) is d(a, 6) = || a — 6 ||. Then, clear! 
lea|=(q¢—D. 


aeGF(q) 
Hence the following results are immediately evident. 


Theorem 5: Relative to the Hamming distance ai 
the Hamming norm, the following results are valid: 

1) The sum of the norms of all the code words in 
systematic, nontrivial code C in V,(q) is oa || @ || 
n/q- |C\+(q— 1). 

2) If d > (n/q) (q — 1), then 


d 


SMO) es 
d—-(q-1)) 
(q 
Theorem 6: Relative to the Hamming norm a1 
metric, the set (8) is a systematic code of size gq” al 
length g” — 1 such that the distance between code wor 
is gq" ‘(q — 1). This code is relativety maxim: 


S.(q” — 1," "(@ — 1)) = q”. 


V. APPLICATIONS TO THE Cycuic DISTANCE 
FOR PRIME FIELDS oF ODD ORDER 


We utilize the cyclic distance of Lee.* Following Le 
we can order the elements of GF(q) on a circle, sep 
rating each pair of adjacent elements by a unit al 
and define the distance, p(a, 8), between pairs of fie 
elements a, 8 as the minimum number of ares betwe 
a and 8. Then the distance between a = (a,)? and b 
(8,)} is 6(a, b) = 55"_, p(a,, B:). Invorder to utilize t 
results of Sections IJ, III and III-A for this distant 
it is necessary that p(a, 8) be translation-invariant relati 
to the addition operation of GF(q). Since the additi 
of GF (q) is already fixed, we can make p(a, 8) translatio 
invariant only by properly ordering the elements 
6, --+ on the circle. In this section we shall show he 
to do this for prime Galois fields. In the following secti 
we shall give the appropriate ordering for GF(2’) a 
shall prove the impossibility of finding a satisfacto 
ordering for other nonprime fields. 

We define a translation-invariant cyclic distance — 
GF (p) (p odd) as follows. First of all, we represent GF( 
by the “absolutely least’? complete system of residu 
modulo p in the ring of integers: 


are) = {0, 41,---, 2254 
with the usual operations for addition and multiplicatic 
We then define the cyclic norm of a in GF(p), | a |, 


| a | = the absolute value of the real integer a. 


The cyclic distance, p(a, 8) = | a — B | is then translatic 
invariant. For elements of V,(p), the corresponding cyc 
norm and cyclic distance are || b || = >°"_, | B; |, 6(a, b) 
|| a — 6 ||. We then have 

Dy lal = seed 


aeGF (p) 
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ence, from Sections II, III, and III-A we readily obtain 
neorem 7. 


Theorem 7: Relative to the cyclic norm and cyclic 
stance, the following are valid: 

1) The sum of the norms of the code words in a non- 
ivial systematic code C in V,(p) is 


Sletl= (ic - =. 


«If (aay then 
Dp 4 
d 
anes), 
: 2 4 


Theorem 8: Relative to the cyclic norm and distance, 
e set (8), with q = p, is a systematic code of size p”™ 
ngth p” — 1 such that the distance between code words 

~* (p? — 1)/4. Moreover, (8) is relatively maximal; 


ip —1,p"” (p’ — 1)/4) = p”. 


VI. APPLICATION To THE Cyrcuic DISTANCE 
For NonprRIME GALoIs FIELDS 


We first carry out the program of Sections IV and V 
r GF(4); then we show that this same program can 
be applied to other nonprime Galois fields. Consider 
7(2°). Let its elements be 0, 1, w, w where 0 and 1 
e respectively the additive and multiplicative identity 
sments of GF(2”). Special values of operation for this 
Id are: 1 +w+to =0,0 =lata = Oforalla 
nis last rule is a property of all GF(2"). Starting with 0, 
» can place the elements of GF(2”) on a circle in the 
xckwise direction in the order 0, w, 1, w and proceed 
in the first paragraph of Section V to define p(a, 8), ---. 
stead, we proceed directly as follows. Define a norm 
Ber (4) as | 0-| =-0, [ow | = | o | = 1,|1] = 2, and 
e distance p(a, 8) = | a — B |. Clearly p(a, 8) is trans- 
jion-invariant. Hence, defining the norm and distance 
r V,(q) as || 6 || = Dota: | 8 |, 6(@, 6) = Ha — B II, 
obtain 
iene, 


aeGF (4) 


d the following, by now evident, result. 


Theorem 9: Relative to the cyclic norm and cyclic 
tance, the following are valid: 


1) The sum of the norms of the code words in a non- 
trivial systematic code C in V,(4) is 


Dilel 


2) Ifd > n/4.4= 


=7lCl4=n/C|. 
n, then 


Sdn, d) < 


— 
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Theorem 10: Relative to the cyclic norm and distance, 
the set (8), for g = 2”, is a systematic code of size 4” and 
length 4” — 1 such that the distance between code words 
is 4". Moreover, (8) is maximal; S, (4" — 1, 4”) = 4 

We now consider a specific case for which the preceding 
process is impossible. Consider GF(3*), and designate 
its elements by {0 = ao, a, a1, +++ , as, a4}. Starting 
with 0, place these elements around a circle in the clock- 
wise direction in the following order: 0, a1, a2, a3, a4, 
Q@_4, Q@3, 2, @-,, Separating adjacent elements by a 
unit arc. Then the distance p(a, 8) between elements 
a, B of GF(Q) is defined as the minimum number of arcs 
between a and 6. We shall show that p(a, 6) can not 
be translation-invariant; that is, it is impossible that for 
ally, p(a + y, 8 + y) = p(a, 8) where a + y¥ is, of course, 
the field sum. In fact, suppose that p(a, 6) is translation- 
invariant; then we can define the norm of a as | a | = 
p(a, 0). Then | @ | satisfies (n?), (m2), (m3) and (n4) of 
Section IT, and p(a, 8B) = |B — a| = | a — B |. Clearly, 
we must have a; equal to the absolute value of the integer 7. 
The requirement that | a, | = | —a, | = 1, together with 
the fact that the only two elements with norm 1 are 


a, and a_,, implies that either —a, = a; or —a,; = a_y. 
But 2a, ~ 0. Hence a_; = —a,. Similarly, —a; = a_; 
for 7 = 2, 3, 4. The distance between a, and a, is 1. 
Hence we must have | a2 — a, | = 1; hence a, — a, = ay. 
But a. — a, = —a, implies a. = 0, so that we must 
have a, — a, = a. Therefore, a, = 2a,. Similarly, 
we can show that a, = 2a, so that ag = 4a, = a,. Then 
| a, | = | a, |; but this is obviously impossible. Therefore 


p(a, B) is not translation-invariant. The ordering of the 
elements of GF(9) on the circle was arbitrary, since the 
one-to-one mapping of nonzero elements of GF(9) on the 
indices +1, +2, +3, +4 is clearly arbitrary. Hence for 
no ordering of elements of GF(9) on the circle can we 
define p(a, 8) as the minimum number of ares between 
a and 6 and have p(a, 8) translation invariant. The above 
analysis applies with only trivial changes to arbitrary 
GF (p") for p odd and k = 2,3, ---. 

Consider now the field GF(2"), k > 3. Let its 2° distinct 
elements be denoted by 


{0 = Ao, Ay (i ae +1, +2, OD av + (2*-* <a Ae Oa 


Order these elements on a circle in the clockwise direction 
in the order 0, a1, a2, °°° , 
a_;, Separating adjacent elements by unit arcs. Then Henne 
the Lee cyclic distance, p(a;, a;) as the minimum number 
of arcs between a;, a;. Then p(a;, a;) is not translation- 
invariant. Suppose the contrary. Then defining | a | = 
p(a, 0), we have p(a, 8) = |a — 6 | = | 8 — @ |, and 
| a | is a norm in the sense of Section II. Moreover, 


Qlcgk—1—1), Olgk—s, M_(g1—1_3)) °° 


| a; | = |2| = the absolute value of the integer 7. Because 
a; = —a,;, we have a_; # —a; [which is different from 
the case GF(p‘), p odd, treated above]. We have 
| a2 — a, | = 1. Hence ag — a, = a or ay. But a, — 
a, = a, implies a, = 0, which is a contradiction. Hence 


(Gey (OF 9) ORIG (9) 
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In addition, | a; — a | = 1 implies 
Qs — a = B, ** (10) 
where 8 = a, or 8 = a_,. Adding (9) and (10) we obtain 
ae nec mary a Ci) 


If 8 = a-,, then a; = a,, which is impossible. If 6 = a,, 
then a3 = a_,, which is impossible. Hence p(a, 8) can 
not be translation-invariant. We summarize the above 
results as follows. 


Theorem 11: Let GF(q) be a Galois field of ¢ elements 
where q is either 2" for k > 3 org = p", p odd and k > 2. 
Order the elements of GF(q) on a circle with unit arc 


between pairs of adjacent elements, and define the 


distance, p(a, 8), as the minimum number of ares between 
a and 6. Then p(a, 8) is not translation-invariant in the 
sense of (4). 

In order to extend the results of Section V to other 
nonprime fields, methods other than those of Section II 
are being investigated. 


VII. EXAMPLES For THE Case GF (2”) 


It seems worthwhile to give examples of the preceding 
discussion. Consider the field GF’ (4) with the representation 
and laws of operation given in Section VI. Using the 
sieve method of Eratosthenes and trial and error it is 
easy to show that the oniy irreducible second degree 
polynomials with coefficients in GF (4) which yield M- 


sequences are the following: 

we’ +24 1, 

wr +at4], 

wx’ + wx + 1, 

oe a @ tested 
These correspond respectively to the following recurren 
relations: 


Oy = WO + Op-1, 
—s 2 
Ay = W Ayo = Qk-1) 
Ay = WAz-2 + WAy-1, 
2 2 
Ay = WA. + Ww Oy-1. 


As an example, we consider the relation a, = waz,-2 + a, 
which yields an M-sequence of period 4” — 1 = 15 fre 
which we obtain the set (8) given by 


@=0 0 0.0 0 °0.0°0. 0 0-00" tea 
a =1 1 @ 1.0. oe 1. One soe 


a@=0 1°10 1 0 & wo 1 wo Oo @ oon 


We can then show, for instance, that a4, — a, = 
and that 
d(a,, @) = 12 = (4 — NY i 
i(a, G2) = 164 


where d(a, b) is the Hamming distance, while 6(a, b) 
Lee’s cyclic distance. 
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A Partial Ordering for Binary Channels” 


N. ABRAMSON}, MEMBER, IRE 


Summary—The properties of iterated binary channels are investi- 
ited. An ordering (defined by the symbol >) of communication 
tannels with two possible inputs and any number of possible 
itputs is defined. For any two such channels C, and (C., this 
dering has the property that if C, D C2, the minimum average 
ss when using C, will be less than the minimum average loss 
ing C», independent of the losses assigned to the various errors, 
1d independent of the statistics of the source. This ordering is 
plied to 1) the general binary channel, 2) the iterated binary 
mmetric channel, and 3) the unreliable binary symmetric channel 
ken used with many iterations. Curves allowing one to use the 
dering are given, and an example using these curves is worked 


st. 


I. Comparison or BINARY CHANNELS 
. Introduction 


7 \ ONSIDER the use of a noisy binary channel to 
GF transmit information with a high degree of relia- 
bility. There are two common solutions to this 
roblem-—block coding and iteration. In this paper we 
sfine an ordering of binary channels pertinent when they 
re used with iteration. 
Let a binary source emit symbols at the rate of one 
er second with the probability of a zero equal to w, 
ad the probability of a one equal to (1.— w). Let us 
efine a channel C by the transition probability matrix’ 


hore [ 1—q| 
Pp i 


(1) 


id let C’ transmit at the rate of m binary symbols per 
cond. We may then transmit the message symbols 
om the source by iteration; that is, by transmitting 
oups of either m zeros or n ones through the channel C. 
Then used in this manner, we may think of the channel C 
; generating another communication channel C,. C,, is 
channel with two possible input messages—n zeros and 
ones—and 2” possible output messages—the set of 
| m digit binary numbers. We shall refer to the channel 
_as the nth-semi-extension of C. 


_ A Sufficient Statistic for C, 


Let us call the two possible input messages to C,, 
(n zeros), and s, (n ones). We also define the 2" possible 
itput messages aS 2, 21, °** Zen-1 where 2; corresponds 
_the n digit binary number 7. Then it is possible to 


-* Received by the PGIT, December 14, 1959. The work described 
this report was supported by the Office of Naval Res., Contract 
onr 225(24). f 

+ Stanford Electronics Labs., Stanford, Calif. ) ; 

1 q is the probability of receiving a zero, if a zero is transmitted, 
d p is the probability of receiving a zero if a one is transmitted. 
e sha] assume throughout the paper that g > 3 and p < 3. 


write a 2 by 2” transition probability matrix for C,,, 
Py Pu Py ony 
I Ps zen P, 27—1 


(2) 


where P;; is the conditional probability of z; given s;. 
P,; is given by the equation 


1 an wi N—Wj eS 
a sao p)”i(p N-Wi 4 = 2 
where 
w; = number of ones in 2;. (4) 


In order to decide which of the s; was sent from an 
inspection of the received message, it is clearly not 
necessary to know exactly which of the z; was received. 
It is necessary only to know w;. That is, w; is a sufficient 
statistic for the determination of the input message. We 
need, therefore, only distinguish n + 1 different outputs 
of the channel C,, — w; = 0, 1, --- n. By considering the 
channel C,, to have two possible inputs—s, and s,—and 
nm + 1 possible outputs—O, 1, 2, --- m—we may then 
write a simpler transition probability matrix as shown 
below. 


or ("aa - 9 (oa - 9a - 9" a 
(p)" ("\e pal) ("ora ey eae teks) 


C. The Decision Theory Problem 


The problem of deciding which of the s; was actually 
sent on the basis of which of the outputs (0, I --- ) 
is received can be treated by the methods of statistical 
decision theory. We define a loss matrix 


Ly, Ly 
Ly, Lee 


L= (6) 


where L;; is the amount we lose if we decide that s; was 
sent, if s; was the true message sent. Then, remembering 
that the a priori probability of s; is w, and the a priorz 
probability of s, is (1 — w), we may compute the optimum 
(Bayes) decision procedure—that procedure which mini- 
mizes the expected loss. Let us call the minimum value 
of the expected loss R,,(w, L). 

Now, if we are given the choice of using C,, the nth- 
semi-extension of © or C/, the mth-semi-extension of 
some other channel C’, it is a simple matter, in principle, 
to determine which channel to use. We need only compute 
R,,(w, L) and R/(w, L) and choose the channel with the 
smaller minimum expected loss. This method will generate 
a complete ordering of all possible semi-extensions of all 
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possible binary channels. This ordering will, in general, 
depend upon w and L. The surprising fact is that it is 
possible to assign a (partial) ordering to the C,, which 
is based upon minimum expected loss, yet which is 
independent of w and independent of L. That is, in certain 
cases, it is possible to say that C, is better than C/ (that 
it will produce a smaller minimum average loss) inde- 
pendent of the statistics of the source (w) and independent 
of the losses we may assign to the different errors (L). 

If R,(w, L) is less than R/(w, L) for all w and all LZ, 
we say that C,, is more informative than C/, written C,, D Cy. 

In the language of statistical decision theory, the 
determination of the transmitted message of C,, from the 
received message is a simple hypothesis testing problem. 
There are two hypotheses: H,, or s; was sent, and Hs, 
or s» was sent. The observation of which of the outputs 
(0, 1, 2, --- ) is received, is said to constitute an experi- 
ment, and the outputs (0, 1, 2, --- m) are called the 
possible outcomes of the experiment. 

It is important to note that by giving the transition 
probability matrix (5), we have completely defined the 
experiment corresponding to C,. Henceforth, we shall 
refer to this matrix as the experiment matrix. The experi- 
ment matrix for the problem of testing hypotheses H, 
and H, is just a2  k matrix (P;;), where k is the number 
of possible outcomes of the experiment. P;; is the con- 
ditional probability of outcome j, given that H; is true. 
The problem of comparison of experiments—or equiva- 
lently comparison of experiment matrices—has been 
treated extensively in the statistical literature.”~° We 
shall follow the treatment of this subject as given by 
Blackwell.” A detailed explanation of this treatment is 
presented,’ and a condensed version is given’ in two 
articles by this author. 

For the sake of completeness, we shall repeat” the 
condition for the comparison of two experiments in the 
next section. 


D. Comparison of Dichotomous Experiments 


Let P be an experiment with k possible outcomes 
(41, V2, +++ %) used to test the hypotheses H, and H,,. 
P may be defined by the two by k matrix 


| 11 | 
21 Pox 


2 D. Blackwell and M. Girshick, “Theory of Games and Statistical 
Decisions,’ John Wiley and Sons, Inc., New York, N. Y.; 1954. 

3D. Blackwell, ‘‘Hquivalent comparisons of experiments,’ 
Ann. Math. Stat., vol. 24, pp. 265-272; 1953. 

4D. Lindley, ‘On a measure of the information provided by an 
experiment,’’ Ann. Math. Stat., vol. 27, pp. 986-1005; 1956. 

°>R. Bradt and S. Karlin, “On the design and comparison of 
certain dichotomous experiments,’ Ann. Math. Stat., vol. 27, 
pp. 390-409; 1956. 

6 N. Abramson, “Application of ‘Comparison of Experiments’ 
to Radar Detection and Coding Problems,”’ Stanford Electronics 
Labs., Stanford, Calif., Tech. Rept. No. 41; July 28, 1958. 

7N. Abramson, “The application of comparison of experi- 
ments to detection problems,” 1958 IRE Natronau CoNvENTION 
REcoRD, pt. 4, pp. 22-26. 


Pre 
P22 


(7) 
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where P;; = P,{x;/H,}; let 
a; = Py; + Pa;; (§ 
define 
Fp) = dia; 
(¢ 


(the notation on the right side of (9) indicates that fe 
any ¢ we sum only those a; such that P;,;/a; < 2); finally 
define 
t 
K,(t) = / F p(w) dw. (16 
0 
Then, if Q is some other experiment with the same numbe 


of hypotheses, but not necessarily the same number ¢ 
outcomes, 


PD Q if, and only if 


K>(t) > Ko(t) for all ¢ in (0, 1]. (11 


As suggested by (11), we shall call K>p(é) the compariso 
function of experiment P. 

From the above definition, it may be seen that if P D & 
then Q C P, read Q is less informative than P. Furthei 
more, if 


PS). Q. cand. O02 
then 
Pes hy, 


That is, the relationship defined by > is transitive 
Finally, note that in general, given any two experiment 
P and Q, we cannot say that either P D Q or QD F 
When neither of these relations hold between experiment 
P and Q, we say that P and Q are noncomparable. I 
other words, the relationship defined by D defines 

partial ordering over all 2 by k Markov matrices,* ¢ 
equivalently over all channels with two possible tran; 
mitted messages and any number (not necessarily ¢ 
same for the channels being compared) of possible a 
messages. 


II. CoMPARISON OF THE GENERAL 
BINARY CHANNEL 


A. The Comparison Function for C,° 


Let us apply the condition given in Section I-D 14 
two simple cases. First, consider the channel C, whe 
n = 1. That is, each signal, zero or one, is sent as it” 
received, and there is really no iteration (or codin; 
at all. The experiment matrix for C, is just the transitic 


8 This ordering may also be extended to incl 
experiments. See footnote 3. ee ae 
® We shall adopt the notation 


F(t) for F¢,(t) 
K,(t) for Ke,(t). 


and 
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obability matrix 


i ems 
" q) (12) 
pep 
From (8), we have’® 
ORC 0 
w2=(1-g+-p), 
d from (9), remembering that g > } and p < }, 
0 tay 
ea Op). fo <i at (13) 
9 t? < t 
dere 
po A pa 
d—@ +d —>p) ee 
, 14b 
ae (14b) 


nally, we may plot K,(¢), the comparison function of 
min Hie. 1. 
The Comparison of Two Binary Channels 


From (11) we see that C, D C{ for any two noniterated 
nary channels C, and C7 if, and only if 


Kae Ki@) for all ¢ in ~ [0,1]; (15) 


d after a little algebra, we see that a necessary and 
ficient condition for (15) to hold is 


: 7? < po?” (16a) 
d 
pt = p20", (16b) 
nally, using (14), we may express (16) as 
: Dees 
ee Le 
q q Ce) 
d 
ik = lL — p’ 
at P.. (17b) 


he nige pe Lng 


Hig. (17) may be expressed graphically as in Fig. 2. 

Any channel C, with experiment matrix as in (12) 
wy be thought of as a point in the unit square. Strictly 
aking, since we have assumed that g > 4 and p < 3% 
should only consider points in the upper left-hand 
arter of the unit square. In Fig. 2 we have fixed the 
int (p’, q’) corresponding to the channel Ct. The set 
channels whose parameters p and gq satisfy both (17a) 
d (17b) will then lie in the shaded region of Fig. 2. 


10 This simple example, until (16), is taken with some notational 
mges, from Blackwell (see footnote 2). It is repeated here for 
lagogical reasons. 
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Fig. 1—Comparison function for the general binary channel. 


(ep, 4‘) 


a 


° 0.5 1 
p— 


Fig. 2—Comparison of C, and C,’. 


These channels are all the channels which are more 
informative than C’. The set of channels whose parameters 
p and q satisfy (17a) and (17b) with the inequality signs 
reversed will lie in the dotted region of Fig. 2. These 
channels are all the channels which are less informative 
than C{. The channels corresponding to the unshaded 
and undotted regions of Fig. 2 are those channels which 
are noncomparable with C{. 


C. The Shannon Ordering 


For the case of the noniterated binary channel, it 
is interesting to compare the partial ordering presented 
in the previous section with a different partial ordering 
discussed by Shannon.’ We shall use the symbol > to 
denote Shannon’s ordering. It is easily shown that if we 
have two channels C, and C/, then C, D C{ implies 
C, > C!. The converse, however, is not true. For the 
case of the binary symmetric channel, it may be seen that 
the two orderings are equivalent. 


III. CompaRISoNn oF THE ITERATED BSC 


A. The Comparison Function for the Iterated BSC 


Now consider the channel C,,, the nth semi-extension 
of C,. The experiment matrix for C, is given in (5). In 
principle we may start with the experiment matrix and 
calculate its comparison function K,(¢). To compare C, 


1 C, Shannon, “A note on a partial ordering for communication 
channels,”’ Information and Control, vol. 1, pp. 390-397; December, 
1958. 
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with C/ the mth semi-extension of C{ we need only examine 
K,() — Kd) 


and note whether this difference is non-negative or non- 
positive in the interval [0, 1]. In practice, however, for 
large n such a procedure can become quite tedious. For 
the case of the nth semi-extension of a binary symmetric 
channel (BSC), however, there does exist an easy method 
of obtaining the comparison function. For the BSC the 
transition probability matrix is 


ee Tle sep 
and the experiment matrix of the C,,, the nth semi-exten- 
sion of this channel is just. 


(18) 


(1 =p)” @a = ee ("Jaa ip) Die py 


o (ora-n(*\ora-p--a—py 


(19) 


Now, using the notation of Section I-D we see that 


a =(." ,\a-9@ + 0-9 @r), 


J 
and defining 
ri—P_<] 21 
Poae (21) 
we may write (20) as 
a; = (, . a Pas p)"(r' JG et 
gl 2 1) ad) 
Further, as suggested by (9), we define 
n n— 7+ i— 
(i ms a cel | paaiaia (ie 
i= (23a) 
a; 
j= (les Bee nts 1) 
and, simplifying (23a), we obtain 
(ee Fee ene ee 
ee He IEE ji » 2) n : ) 


Now the subscripts on #; and a; in (22) and (23) refer 
to the jth column of the experiment matrix. It will be 
convenient to re-lable the ¢; and a; as follows: 


(n+1) (n+1) 


ty =>. t ay 7 (8! 
(n) } 
Lips = t = (O'by are a 
(26) 
(1) (1) 
ae] => f One ae 
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so that (23b) may be rewritten as 


1 


£2 oe, 


(23 


Note that now (since r < 1) 
O <b te eee 


so that from (9) we may write 


n+1 
(remember take Spe 2 2) 
j=1 
0 Oye 
Re hoe a t a i? 
pce) Sy fP <tc 4 
F(t) EPPA? re Seen 4p <p 2 pe. (2 
9 — ane go << t “2 gery) 
9 PRED << t < 1 


And the comparison function may be written as 


K,(t) = ‘) Eanes 
= Lae) (2 
Oy 
for. tf” << 
and j= 0,1, 2, -:-”=k1 
where we define t‘” = Oandé"*” = 1. 


Note that K,,(¢) is also a function of r. When we wi 
to emphasize this dependence upon 7 we shall wr 
K,(t) as K,,,,(). t isa function of m and r. When > 
wish to emphasize the dependence upon n we shall wr 
t*? as ty). When we wish to emphasize the depende 
upon both n and r, we shall write t<?. The first s 
script will always refer to the number of iterations, 
second to the quantity p/(1 — p). 


B. Four Lemmas 


Iiq. (29) provides one possible method of comput! 
the comparison function for the nth semi-extension | 
a BSC. In this section we shall state several lemni 
which may be used to decrease considerably the amow 
of work involved in this computation. We leave the pra 
of these lemmas to the appendix. 


Lemma 1: Let K,,,,(¢) and K,,,,.(¢) be the compari: 
functions of the nth and mth semi-extension of the BS* 
C,,, and C,,,, respectively. 

If | 


Tea (0) Ss Kn,r(b) for tin [0, 2] 


60 
en 


IGE esi dye tors bine Oad 4: 


Lemma | states that in order to compare iterations of 
3C’s, we need only compute the comparison functions 
BO <i <5. 

Next note the form of K,(t) as given in (29). K,(b) 
a piecewise linear function of ¢ in the interval [0, 1]. 
thermore K,(¢) is continuous in this interval, while 
(t), the derivative of K,(t), is discontinuous at the 
ints #, £, --- ¢”*. We shall refer to these points 
the ‘“‘breakpoints’’ of K,,(#). 

The breakpoints of K,,(¢) are easily obtained from (23c). 
addition by inspection of (23c) we may immediately 
:te Lemma 2. 


Lemma 2: Let K,(¢) and K,_.(t) be the comparison 
actions of the nth and n — 2 th semi-extensions of 
+ same BSC. The set of » + 1 breakpoints of K,(#) 
nsist of 


a) the set of nm — 1 breakpoints of K,,_,(¢), 


1 
»)) 1 + r” ? 
e) meee 1 4 


Lemma 2 states that if we have the n — 1 breakpoints 
K,,_2(t), it is only necessary to compute one additional 
sakpoint to obtain the n + 1 breakpoints of K,(¢). 
mma 2 suggests that a simple method of obtaining a 
mparison function for large n might be to start with 
aller values of n and to proceed by induction. This 
leed will be the method we shall use but first we need 
o additional results. 


Lemma 3: If K,(#) is the comparison function for the 
1 semi-extension of a BSC then, for n an odd integer 


a a . 
: E KO |, Sons 
The last result we need before we show how to obtain 


s comparison function for the nth semi-extension of 
35C is Lemma 4. 


Lemma 4: Let K,(t) and K,_,(¢) be the comparison 
retions of the nth and n — Ith semi-extensions of 
» same BSC. Let #32, #, --- #*” be the n + 1 break- 
ints of K,,(¢). Then 


Tee = Gt”) 
[Ol i=l oe 


ai) =cly2s 0” 1) ant 


Lemma 4 tells us that K,(t) will equal K,-,(¢) at the 
akpoints of K,,(t). Thus, if we know K,_,(¢) we know 
. value of K,(é) at its breakpoints. K,,(t) 1s, however, 
inear function of ¢ between any two successive break- 
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points so that the values of K,(é) at its breakpoints are 
sufficient to determine the function everywhere. 


C’. Construction of the Comparison Function for an 
Iterated BSC 


We shall now illustrate the application of Lemmas 1, 
2, 5 and 4 to the construction of a comparison function. 
As an example, let us say we wish to construct K,(t) for 
n = 10 and r = 0.5. Note that by Lemma 1, we are only 
interested in K,(¢) for 0 < ¢ < 4. Now the breakpoints 
of K,(¢) and K,(t) may be computed from (23c). In 
Table I we have listed for these two comparison functions 
all the breakpoints which occur in the interval [0, 3]. 


TABLE I 
BREAKPOINTS OF K,)‘*) aND Kg‘*) rorr = 0.5 

Ko K,(») 
t@) 0.001 0.002 
t@) 0.004 0.008 
t@) 0.150 0.300 
t@) 0.590 0.111 
t®) 0.200 0.333 
t 6) 0.500 


By Lemma 2 the breakpoints of K,(¢) occurring in the 
interval [0, 3] may be obtained by dropping the first 
entry in the first column of Table I. The breakpoints 
of K,.(¢) may be obtained by dropping the first two entries 
in the first column, etc. The breakpoints for K,(t), K;(#), 
K;(t) and K,(¢) are obtained from the second column 
in Table I in the same manner. 

The first step in the construction of K,)(¢) then is to 
indicate the breakpoints of K,(¢), K2(¢), --- Kyo(t) in 
the interval [0, 3]. These breakpoints, which by Lemma 2 
consist of the eleven numbers in Table I, are shown by 
vertical lines in Fig. 3. 

Having drawn these lines, it is a simple matter to 
construct K,(¢) for t in [0, 3]. The only breakpoint of 
K,(t) in [0, 4] is the last entry in the second column of 
Table I (¢ = 0.333). By Lemma 3, K,(t) must have a slope 
of unity at ¢ = 3. We therefore draw the line starting 
at this breakpoint with unit slope. This line is equal 
to K,(é) for 0.3383 < # < 0.500. K,(é) is equal to zero 
for 0 < ¢ < 0.333. K,(¢) may now be drawn immediately 
as shown in Fig. 3 by a direct application of Lemma 4. 
In exactly the same manner, by using Lemma 4, we 
construct K,(t) to K,(t) without further calculation.” 

In Figs. 4-6 we have drawn the comparison functions 
for some other values of 7. 


D. Example 


As an example of the application of these figures, con- 
sider the use of the third extension of a BSC with proba- 


2 Tt, is necessary to employ the slope condition given in Lemma 
3 to obtain the last segment of K;“ for odd 7. 


034 


Fig. 3—Comparison functions, K,(t), for 7 = 0.5. 


Fig. 5—Comparison functions, K,(¢), for r = 0.7. 


bility of error equal to 0.375 (r = 0.6). By superimposing 
Fig. 4 on Fig. 5 it may be seen that 


Ks,.6(t) = Kan) OStS 


Nile 


and 
Ks,.0(f) S Kz,.7(0) VES ieee 


That is, if the probability of error of the BSC increases 
to 0.411 (r = 0.7) the first channel when used with three 
repetitions will always be preferred to the second channel 
even if we use it with as many as four repetitions. On 
the other hand the first channel when used with three 
repetitions will never be preferred to the second channel 
when used with eight or more repetitions. 


E. Behavior For Large N 


By inspection of the comparison functions in Figs. 3-6 
(or from Lemma 4) it may be seen that for a fixed r, 
Kit) > K,(t). That is, the n + 1th semi-extension of 
a BSC is more informative than the nth semi-extension 
of the same BSC. This not too surprising fact brings up 
the question of the behavior of K,(t) for large n. Does 
K,,(t) approach a limit and if so is this limit a function 
of r? These questions are answered by Theorem 1. 
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Fig. 6—Comparison functions, K,(¢), for r = 0.8. 


Theorem 1." Let K,(#) be the comparison funct: 
of the nth semi-extension of a BSC with transition prol 
bility matrix 


ee p | | 
p Lisa 
a lt p< § 
lim K,(i) = t OS eels 
b) ifp = 3 
1 
im Ko = One Stee 
iy of = 1 ea 


We have plotted these two limits in Fig. 7. We s: 
call the greater of these limits the maximum compare 
function (MXCF) and the other the minimum compare 
function (MNCF). It is not possible for a channel v 
two inputs and any number of outputs to have a ¢ 
parison function greater than the MXCF at any po 


in [0, 1]. Likewise, it is not possible for a channel y 
two inputs and any number of outputs to have a c’ 


18 Theorem 1 is proved in the appendix. 


960 


Maximum comparison 
function 


Minimum comparison 
function 


3 zr 
os 


Fig. 7—The maximum and minimum comparison functions. 


arison function less than the MNCF at any point in [0, 1]. 
‘he MXCF corresponds to a channel which can transmit 
inary information with zero error. The MNCF  cor- 
ssponds to a channel where the output is statistically 
1dependent of the input. 


IV. Comparison oF THE ITERATED BSC Wirx C/ 
.. [wo Comparison Theorems 


In this section we shall investigate the comparison of C,, 
ae nth semi-extension of one BSC, with C/, some other 
oniterated BSC. By means of such a comparison it will 
e possible to compare iterated BSC’s in general. That is 


Ue aCe Cy 
anon tie Cres 
aen we know that 
CRD .C a: 


The reason for comparing two iterated BSC’s through 
1e artifice of introducing another, noniterated BSC may 
e seen in the following lemma.™ 


Lemma 5: Let C, be the nth semi-extension of the 
SC C, and let C? be some other BSC. Then 


: ma) Cl > C,, if and only if 


ee Sod ase (30) 
Bo) C, . Clif and-only if 
KEG) Pe Hig, (3) (31) 


We are now in position to state the two central results 
? Section III. First, by using in (80) the expression for 
? given in (23c), and then writing the result in terms 
f p, we obtain Theorem 2, as follows: 


Theorem 2: Let C, and Cf be two BSC’s with proba- 
ilities of error p and p’ respectively. Let C, be the nth 
mmi-extension of C’,. Then 


Ci > C, if and only if 


(p)" 
Qc sleep) 
Next we note that for n even #/** = 3, so that 
y Lemma 4, K,(3) = K,-1(3), again for even n. Then 
» get K,_,(3), we use (29), setting ¢ = 3, and finally 
otain: 


jae (32) 


“4 Lemma 5 is proved in the appendix. 
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Theorem 3: Let C,, and C{ be two BSC’s with proba- 
bilities of error p and p’ respectively. Let C, be the nth 
semi-extension of C,. Then 


a) for n odd 
C, > Cif and only if 


Deel 2 Nord ae aancen 


b) for n even 
C, > Clit and only if 


Car D Ch. (33b) 


B. Some Comparison Regions 


In most of the remainder of Section IV we shall be 
interested in investigating the properties of highly un- 
reliable BSC’s when used with a large number of itera~ 
tions. Accordingly, we shall find it convenient to define 
the BSC parameter « by 


p= 370 -2@) 


where p is the probability of error of the BSC. We recall 
that 


(34) 


ae Pp 
ee eas 5 (21) 
so that 
l—e 
a rae (35) 


For a highly unreliable BSC the BSC parameter ¢ will 
approach zero. 

Let ¢ and e’ be the BSC parameters of C, and C{ re- 
spectively. Then (32) may be written as 


Ci), CG, it-and onlyait 
(Gere! 


1-—é 


(ee eee Cpa) 
or 
CFS Crinand.only ir 
Gea D iter = 
e+ e + e + 
Memeo gee ULE NU os. (36b) 


i nN\ 2 n\ 4 
1+ (3). +(‘). cee 


where we have defined 


(") =o for m>n. 
m 


Likewise, we may rewrite (33a) in terms of ¢ and é’ 


as follows: For n odd 


C2 Chl andonly.at 


eae) > OG oD 


k=1 
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In Figs. 8-11 we have plotted in the e¢, e’ plane regions channel capacity of C{ is, of course, 
where C{ > C,, and where C,, D C! for n = 3, 5, 7 and 9. ‘ ; ' thy ; 
Also included in these figures is the equicapacity line— Ttep! log pit lies a) slog (aad 2 @ 
the line which gives for any e’ the value of ¢« such that It is interesting to note that the equicapacity li 
the Shannon channel capacity of C, (in bits per unit lies in the noncomparable regions of Figs. 8-11 exce 
time) is equal to that of C/. The channel capacity of C, for a small range of values of ¢’ corresponding to high 
is given by reliable BSC’s. This range of values vanishes rapidly 
n increases. In this range, however, we have the i 
ne ep eee lob (38) teresting phenomenon of two channels C, and C’, wi 
where we have multiplied by n so that one digit from the channel capacity of C, greater than that of C’ ai 
the source may be transmitted by C, (using iterations) yet with the average loss using C,, (in an iterated manne 

in the same time that it may be transmitted by C{. The always greater than that of C’. 
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Fig. 8—Comparison of the iterated BSC with the noniterated BSC. Fig. 9—Comparison of the iterated BSC with the noniterated BS 
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Fig. 10—Comparison of the iterated BSC with the noniterated BSC. Fig. 11—Comparison of the iterated BSC with the noniterated B 
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. Iteration of Unreliable BSC’s 


One final question of interest is the behavior of highly 
areliable channels when used with many iterations. That 
, we are interested in comparing C,, with C/ when e 
oproaches zero and 7 is a large number. 

Under these conditions (36b) reduces to 


Cl > Gif and only if 


, 
€ 
oa: 


= (40) 


We may also let ¢’ approach zero in (37). After a good 
eal of algebraic manipulation and the application of 
tirling’s formula, this will reduce to 


CDG? Mf and*only,if 
, 

yee =: 
2V/n 


Finally from (38) and (39) we obtain (again for small e): 
The capacity of C, [from (38)] is equal to the capacity 
[ Ct [from (39)] when 


(41) 


(42) 


We may indicate the results of (40)-(42) as shown in 
ig. 12. 


No. of iterations = 


equicapacity 
line 


e — 
g. 12—Comparison of the iterated unreliable BSC with the 
noniterated BSC. 
APPENDIX 


ome Proofs 


Lemma |: 


Let K,,,() and K,,,,-(¢) be the comparison functions 
the nth and mth semi-extensions of the BSC’s C,,, 
id C,,,- respectively. 
If 

K,,, (0) = Kn for ¢ in [0, 2] 
en 


Ke Ales og eh). for, “bin, {0, 1). 
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Proof: From (10) 
K,() = [ Palw) dw (43) 
0 
and letting 
v=l1-w 
1 
KO = [Ful - 9) de, (44) 
1-?¢ 


but from (28) we see that if K,,(¢) is the comparison func- 
tion of the nth semi-extension of a BSC, then 


F,(1 — v) = 2—F,(v) for v in (0, 1] (45) 
except on a finite set of points, so that 
1 
KW = [ 2 - FO] a 
de (46) 


2b KD) a) 


(24-1) + K,(1 — 2). 
Lemma | then follows directly from (46). 


Lemma 2: 


Let K,(t) and K,_2(t) be the comparison functions of 
the nth and n — 2th semi-extensions of the same BSC. 
The set of n + 1 breakpoints of K,,() consist of 


a) the set of nm — 1 breakpoints of K,_.(¢) 


1 
») 1+r” 
1 1 
y [oe see eres 
Proof: Lemma 2 follows directly from (23c). 
Lemma 3: 


If K,(t) is the comparison function for the nth semi- 
extension of a BSC then, for n an odd integer 


d 
E Ko] = 1. 


Proof: For n odd we may use (29) where j = (n + 1)/2. 
Then, taking a derivative with respect to ¢, we get 


d (n+1)/2 
E Ko | = aes 
dt t=1/2 Ral 


Finally, from (19) we see that the right side of (47) 
is Just one. 


(47) 


Lemma 4: 


Let K,(¢) and K,-,(¢) be the comparison functions of 
the nth and n — Ith semi-extensions of the same BSC. 
Let #, #, --- £*? be the n + 1 breakpoints of K,,(Z). 
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Then 
K, (tr) = Kani’) 
for n = 2,3, °°: 
f= 12 eh ade 
Proof: The first step is to note that 


and 


Ke (Gea eae 2) a0 (48) 
next we show that 
KG”) = eis ys (49) 


K,,(¢) is just the integral of /’,(¢). In Fig. 13 we have 
plotted the first section of F,,(t) and I’,_,(é). Referring 
to this figure we see that K,(t{”) is just the area under 
F,(¢) from 0 to # while K,_,(é7) is the area under 
F,,_,(¢) in this same interval. Thus to show that these 
two areas are equal we need only show that the shaded 
area A, in Fig. 13 is equal to the shaded area B,. That is, 
we must prove that 


ee io eae et ile a] (60) 


Substituting for the #{” and a{” in this equation will 
then prove (49). 

We might continue in this manner and show that areas 
A, and B, of Fig. 13 are equal so that K,(¢) and K,_,(é) 
are equal for tf = ¢. It is easier, however, to tackle 
the general problem immediately. That is, referring to 
Fig. 14, we shall show that the fact that area A;_, is 
equal to area B,_, implies that area A; is equal to area B;. 

In terms of the quantities in Fig. 14 we wish to show 
that if 


[ale = tA) = 


then we must have 


fat? — L,1eis? — &-] G1) 


We? = 42a = af? + Loa 
See eet We, 1 Ly). (52) 
From (51) we may obtain an expression for L;_, 
Dos fee ga) 
Ly = G4 a1 } (58) 


G 
Up ar 


after substitution of (53) in (52) and a good deal of 
algebra we obtain 
ei tates is 


(i) Gaby F 
tips —; Ga a 


(7 +1) (7) 

Gi aces ree es 

Aa, = An-1 par) G) 4 n—1 
nm — th, 


(54) 
finally, we substitute for ¢{” and a‘ and verify (54). 

Let us summarize what we have done. First we have 
shown that 


Ka) eet) (55) 


for 7 = 1 and 2. Kq. (55) would hold for all 7 if we could 
prove that the areas we have labeled A; and B; were 
equal. Finally we proved that A; = B;, if A;_, = B,;_1: 
since we had already proved A, = B, this completed 
the proof and (55) holds for all j. 
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Fig. 13—The first section of F(t) and Fn_;(t). 


t 
and F (+) 


Lp 
ere Pama, 


ae | 
Fig. 14—A middle section of F',(¢) and F',_,(t). 


Theorem 1: 


Let K,(¢) be the comparison function of the nth sem 
extension of a BSC with transition probability matr’ 


ae p 


jee ey 
a) ifp <4 
limK,() =t 0O<t<1, 
b) i p== 
i 
lim K,(1) =| OStS2 
a 27 sara 


960 
-roof: From (29) we have (for n odd) 


(n+1)/2 (n4+1)/2 
IG = De ga OY. (56) 
| k=1 
But for n odd 
(n+1)/2 
ae! (57) 
nd 
a (k) 4(k) oy? n 
Pe (a -oto.  & 


The sum on the right of (57) is just the probability 
hat a random variable n, having a binomial distribution 
* mean (n)(1 — p) and variance (n)(1 — p)(p) will 
se less than n/2. If p < 4, (n)\(1 — p) > n/2 and we 
nay write (57) as 


n+1)/2 n 
1 k 

a { = Pie <2 

k=1 2 


<P,{|2—- @1 —p)| > @G — v)} 


(p)(1 = p) 
ee ieee 
eG p: 
Where the last step is obtained from the Bienayme— 
“chebycheff inequality. Now we use (56) and (57) in 
55), let n approach infinity, and we obtain (for n odd 
md p < >) 


(58) 


lim K,(3) = 3° 


no 


(59) 


Furthermore, since for a fixed r, Crs: D C,, (59) must 
lso hold for even values of n. Finally, since K,(0) = 0 
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and K,(1) = 1 for all n, and d/dt K,,(t) is a nondecreasing 
function of ¢, (59) proves part a) of the theorem. 

Part b) is then proved simply by noting that if p = 3, 
we have r = 1, and the only breakpoint of K,(t) is at 


t= 4. 


Lemma 5: 


Let C, be the nth semi-extension of the BSC C, and 
let C{ be some other BSC. Then 


a) C{ > C, if and only if 
ies 

b) ¢, D Cf if and only if 
UG ©) pean Gued s 


q) 
t 


N,T) 


Proof: From Lemma 1 we see that we need only prove 


a) Ki) = K,@) in [0, 3] 
if and only if 
‘7 < p® 
b) K,() = Ki) in (0, 3] 


if and only if 
Ke) > Ky»). 


Note that é{?, is the only breakpoint of K{”(é) in 
[0, 4] so that by Lemma 3, the slope of K{”(#) is unity 
in the interval ¢{?, < t < }. The slope of K,(#), however, 
is never greater than unity in this interval (again by 
Lemma 3). Parts a) and b) of Lemma 5 follow directly 
from these two facts. 


In the Probability Density of the Output of a Low-Pass 
System When the Input is a Markov Step Process" 


W. M. WONHAM} 


Summary—Forward equations are derived for the (N + 1)- 
imensional Markov process generated when a Markov step signal 
(t)} is the input to an Nt -order system of the form dX/dt = 
(X; s). As examples, the joint probability densities of input and 
itput are found for a symmetric three-level signal smoothed by 
1 RC low-pass filter, and partial results are obtained for a doubly 
tegrated Rice telegraph signal. 


* Received by the PGIT, January 8, 1960; revised manuscript 


ceived, June 2, 1960. j : 
+ Dept. of Engrg., University of Cambridge, Cambridge, Eng. 


INTRODUCTION 


F THE INPUT to a (possibly nonlinear) system is 
a Markov process, and the system belongs to a 
certain class of functionals, an equation for the 
characteristic function of the output distribution can be 
written using the methods of Darling and Siegert.’ 


1D. A. Darling and A. J. F. Siegert, “A systematic approach 
to a class of problems in the theory of noise and other random 
phenomena—I,” IRE Trans. on Inrormation TuEory, vol. IT-3, 
pp. 32-37; March, 1957. 
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McFadden adapted the Darling-Siegert method to linear 
systems when the input is a Rice random telegraph 
signal,’ and has developed techniques. for other binary 
processes.’ An alternative formulation of the problem, 
for Markov step inputs with a finite number of states, 
is given below in terms of forward equations for the 
joint densities of input and vector output. The step 
process, of which the Rice signal is a special case, arises 
in engineering applications as a model for discontinuous 
velocity signals.* Joint densities of input and output are 
of interest, for example, in the analysis of feedback 
control systems. 


Markov Strep Processes’ * 


Let {s(t)}, 0 < # < © be a Markov process with a 
finite number of states 7 = 1, --- , m. To each state 
corresponds a bounded amplitude level s,, where the s; 


need not all be distinct. Stationary transition probabilities 
Pi i (r) ’ 


p(t) = Prs(t+ 7) =s3,|s)=s], 7120, (1) 
are given by 
P(r) = [pil] 
= exp (7M) (2) 
=[+7M+0(7) as +r 0. 


Here J is the unit matrix, and M = [y,,] is a constant 


matrix such that 


Bij 2 0, 1A a (3) 


ee (4) 


14t 
ay 0 


If s(t) = s;, either a jump transition to s;, 7 ¥ 7, occurs 
in a small time interval (¢, ¢ + 7), with probability 
u;;7 + o(7); or no jump occurs, with probability 1 — p;r + 
o(r). It can be shown’ that the probability of no jump 
in an arbitrary interval (¢, ¢ + 7), conditional on s(#) = s,, 
igo. 

When limiting probabilities p;;(©) = p; exist which 
are independent of the initial distribution, the p; are 


2 J. A. McFadden, “The probability density of the output of 
a filter when the input is a random telegraphic signal: differential 
equation method,’? IRE Trans. on Crrcurr Turory, vol. CT-6, 
pp. 228-233; May, 1959. 

Cid agate McFadden, “The probability density of the output of an 
RC filter when the input is a binary random process,” IRE Trans. 
ON ANFORMATION Tueory, vol. IT-5, pp. 174-178; December, 1959. 

4H. M: James, N. B. Nichols, and R. S. Phillips, ‘ ‘Theory of 
Servomechanisms,”’ McGraw-Hill Book Co., Ine., New York, 
INGE y cleo: 1947. 

5 A, Kolmogorov “Analytische Methoden in der Wahrschein- 
lichkeitsrechnung,’’ Math. Ann., vol. 104, A re pp. 415-458; 1931. 
Syst. Tech. J., vol. 23, article 2.7, p. 282; 1944 

6M. S§. Bartlett, “An Introduction to Stochastic Processes,” 
Cambridge University Press, Cambridge, Eng., ch. 3; 1955. 
7 J. L. Doob, “Stochastic Processes,’’? John Wiley and Sons, 


Inc., New York, N. Ye (cht Gal953: 
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Diki 


= oy Pika; 
ini ( 
ye Di = les 


A familiar example is the Rice random telegrar 
signal:> s(¢) = 8, or 82, and 


at | 1 


where u is the expected number of jumps [zeros of s( 
in unit time. 

In the following section, general equations are derive 
which apply to a fairly wide class of time-invarial 
systems with Markov step inputs. To illustrate tl 
derivation, Example 1 is treated in complete detai 


FoRWARD EQUATIONS FOR THE JOINT 
Input-Output Process 


Let the s(¢#) process be the input to a time-invariat 
system with vector equation 


am = o| x;s00 | ( 


X() = [x'(), --- 2%], U= uw). = 


The system is “low-pass” in the sense that no derivative 
of s(t) appear in (6). It is assumed: 1) that if s¢@ + 7) = 

| > | < T, then (6) yields a unique continuous pha; 
trajectory 

X(t “+; 7) a Flr; X(t), s;] 


Fs(77 XU eer, 


where 


lee"; oe 


(; 


p S15 ee 


and 2) that X(¢) is continuous at jump points of s(é), t.« 


| F,(—7; X) — Fi(7; X) | 0 as 


7 —> 0, 
j,k = 1, ><" in 


From (8), X(¢ + 7) is completely determined for 7 >} 
by X(t), together with s(¢ + 7’) for all 7,0 < 71’ < 
Since {s(¢)} is a Markov process, the conditional dist: 
bution of s(¢ + 7’), 7’ > 0, given s(t) and s(t’) for sor 
t’ < t, depends only on s(t). Hence the conditional dist7 
bution of the pair [X(t + 7), s(¢ + 7)], given [X(é), s( 
and [X(t’), s(t’)] for some t’ < t, depends only on t 
value of [X(¢), s()]. That is, the joint process {X(é), s(# 
is Markov, continuous in the X component and m-valu 
in the s component. 

Let W;(X, t) dx’ --- dx” = W,(X, t) dX be the pro 
bility that s(t) = s; and X(t) « dX at X, conditior 


*S. O. Rice, “A mathematical analysis of random noise,” 
Syse Lechae: vol. 23, article 2.7, p. 282; 1944. 
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on specified initial values X(0), s(0). Equations for the 
W; may be derived as follows. Consider an arbitrary 
interval (tf, t + 7), where t, r > 0. Then X(t + 7) = X 
and s(t + r) = s, if and only if: 1) s(t) = s,, MD 
F; (— 1; X), and no jump of input occurs in (é,¢ + 7); 
or 2) for some r’ in (0, 7) and some 7 ¥ j the following 
conditions are satisfied: s(t + 7’) = s,, X(t + 7’) = 
F; [— (r — 1’); X], a jump 7 => j occurs in dr’ at t + 7’, 
and no subsequent jump occurs in (¢ + 7’, ¢ + 7). Adding 
the probabilities of events 1) and 2) yields 


Wi, t+ 9) = WR (—r; x), | HSN| 


+ f Wr (-G = 29; %),t4 71 


ta 


eee 


yt 7) ? 
ae | Mise? dz’, 


j=1,- 


The Jacobians allow for a change of volume element 
along the j trajectory (8) through X. The forward equa- 
tions can be derived formally by expanding both sides 
of (9) to terms of o(7), dividing by 7, and taking the limit 
z — 0. From (6) and (8), 


. 


I 


EC X p= FU, s;) Fi 07) 


X — 1U,(X) + o(7), 


(10) 


where U;(X) = [u}(X), --- , w¥(X)] is the velocity in 
phase space when s = s;. From (10), 


WF (—7; X), 1 = WAX, t) 


Sy eee LE ©. Sa 
= Dux) ED + on), 
VjAcs 1, +++ mM. (11) 
Also, 
Hae = det % [x — rui(X) + o(9)] 
= det [3,,— 7H 4 on} (12) 


p= s OD Tee 


(2 
r=1 Ox 


The integral in (9) expands simply to 4;;W,;(X, t)r + o(r). 
On substituting (11), (12), and the last expression in (9), 
the limiting process yields the forward equations 


WX, b + div [U)(X) W(X, 0] 


== pe wi;WA(X, t) 3 wiW (x, t), 


fa lea yms (13) 
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where 


div [U;W,] = ss d(u;W ;)/dx’. (14) 


Summing (13) on j and using (4) gives the equation 
of continuity 


m 


> eee + div [U,(X)W,(X, oy} =0. (15) 
In (13) and (15), W;(X, #) may be interpreted as the 
density of j-type gas at the point X, where U;(X) is the 
corresponding drift velocity, and w;;W; is the rate of 
‘conversion’ of z-type gas to j-type. Eq. (13) is analogous 
to Boltzmann’s equation for a mixture of m gases” (with 
particle ‘‘conversion”’ in place of collisions), and might 
have been derived alternatively by an argument based 
on conservation of flow. 

If the system (6) is one-dimensional (NV = 1), (13), 
together with s(0) = s,, W,(X, 0) = 6;,6(X — X)), 
defines a hyperbolic (wave-type) initial value problem 
which has been solved for the Rice telegraph signal in 
special cases.’°'* If N > 2, the system is no longer in 
general hyperbolic, but may be parabolic ‘in the broad 
sense.””’” Boundary values can be obtained by an iterative 
procedure as illustrated below in Example 2. 

The equilibrium densities, when they exist, satisfy (13) 
with the time derivatives omitted. If N = 1, X = 2, 
there results the ordinary system 


£ Us) =D ws Wi) — wWi0), 


da 
tft 
f= 1h s,m, (16) 
In this case, integration of (15) yields 
>> U,(2)W (x) = constant = 0. (17) 
j=1 


That the constant of integration is zero is plausible on 
physical grounds, since the sum represents total ‘“proba- 
bility flow’ along the « axis. For equilibrium densities 
to exist, the flow must vanish everywhere. On substituting 
(17) in (16) one of the unknown functions may be elimi- 
nated. Eq. (17) was suggested by the special case when 
m = 2 and the system is an RC low-pass filter. The 
latter result was pointed out to the writer by Dr. Mc- 
Fadden,** who derived it using a level-crossing argu- 
ment.“ 


9 J. H. Jeans, ‘Kinetic Theory of Gases,’’ Cambridge University 
Press, Cambridge, Eng., ch. 9; 1940. 

10S. Goldstein, ‘On diffusion by discontinuous movements and 
on the telegraph equation,” Quart. J. Mech. and Appl. Math., vol. 4, 
pp. 129-156; 1951. og 

u W. M. Wonham, ‘Transition probability densities of the 
smoothed random telegraph signal,’ J. Electronics and Control, 
vol. 6, pp. 376-384; 1959. ; 

27. G. Petrovsky, “Lectures on Partial Differential Equations,” 
Interscience Publishers, Inc., New York, N. Y., ch. 1; 1954. 

13 J, A. McFadden, private communication. 

4 J, A. McFadden, ‘The probability density of the output of a 
filter when the input is a random telegraphic signal.” 1959 IRE 
Cia ConvEenTION Recor, pt. 4, pp. 164-169. [Cf. (22) and 
23).] 
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Example 1: First-Order RC Filtering of a Three-Level Step 
Signal 


Let {s(é)} be a symmetric three-level. step signal, with 


$; = —s,~= 1, and s = 0. The transition matrix is 
P(r) = [pa(?)] as 
= I — 7M + o(7), 
where 
—l L-@ . 2a 
Melee) =| 8 eee a, 
Cae et De eek Dar aS 
BB S20; Osa. (19) 
From (5) and (19), the limiting probabilities are 
Pi = Ps = B/(1 — @ + 28); 
= (1 —a)/(1 —a + 28). (20) 


On calculation of P(#) = e™’, the covariance function is 
found to be 


R(r) = E{s(ds(t + 7)} 
a 8:P:Pii(7)8; 


—pw(1l+a)|7I 


= 2p.e (21) 


The equilibrium densities will be found when s(f) is 
the input to a first-order low-pass RC filter with RC = 
and differential equation 


dx 
‘dt 


If s(¢’) = constant = s; in the interval ¢ < #’ < ¢ + 7, 
(22) may be solved for x(¢) in terms of x(t + 1): 


x(t) = e' [ze + 7) — 8,] + 8;. (23) 


Let W;(z, t) dx be the joint probability that s(t) = 
and x < x(t) < « + dz {conditional on specified initial 
values s(0), z(0)]. For an arbitrary interval (é, ¢ + 7), 
one has s(¢ + 7) = s; and x(¢ + 7) = 2, if and only if: 
1) 3) =ss,, 40) = “2=-¢ (@ = s)).-- \s;, and no jump 
of input occurs in (¢, ¢ + 7); or 2) for some 27 # 7 and 
some 7’ in (0, 7) the following conditions are satisfied: 
s(é + 7’) = s,, 26 + 7’) = 2 =e" (a — 8;) + 8;, 
a jump 7 — j occurs in dr’ at ¢ + 7’, and no subsequent 
jump occurs in (¢ + 7’, é + 7). Adding the probabilities 
of events 1) and 2), 


Wi, t+ 7) =e W(e1, )(dr,/da) 


= s(t) — x. (22) 


Ef Wila’, t+ r’\(dx? Jane, dr’. — (24) 


On differentiating both sides of (24) with respect to 7, 
and setting r = 0, there results the following special 
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Decembe 
case of G13): 


OW (x, t) 
Ot 


= ae MiiW (a, t) = uj W (a, oF 


ate = [(s; — 2) W;(@, 0] 


j= 1, 2,3. (25 


Eqs. (25) are the forward equations for the joint input 
output process {s(¢), x(#)}. 

It is reasonable to assume that equilibrium densitie 
W (x) exist in the limit t > ©. Since | s(#) | < 1, iti 
seen from (23) that | x(#) | < 1, so W;(z) = 0,|2|>1 
Dropping the time variable in (25) and _ substitutins 
values of the s; and y,; gives 
(d/dx)(1 — «)W,(x) = p(—W, + BW, + aWs), (26a 

(26b 
—(d/dx)(1 + 2)W3(a) = nlaW, + BW. — Wal, 
Me ee (26¢ 
By definition of W;(x), one has f:, W,(x) dx = p, 
Integrating (26a) from —1 to 1 and using (20) yield 
[1 — 2) Wi@))-. = 


Aszxz—+ 1, d — 2)W,(2) — 0, otherwise W, wouk 
not be integrable. Hence 


W.i(—-1) = 0; (27a 
and similarly | 
WA+1) = Wx(+1) = (27b 
On integrating the sum of (26), there now follows 
Gd 2) Walz) =2W@) 
— (1 + 2)W3(a) = constant = 0: (28 


That the constant of integration is zero is seen by lettin, 
Hi Ae ont 
Eqs. (26) with end conditions (27) can be solved i 


terms of the hypergeometric series 
He = ar, so ee wie], (ag 
Riess wp,| Full a = 28 

1 + =o) 1 tal. (3c 


If |«|> 1, W,(¢) = 0. For | z | < 1, the results a 
Wi@) = CUS Gris) 


IE a2 -(154 12) - oma — 2), 


W(x) = CU — a)(1 — 2°)" — 2), 


(ble 


(3 It 
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V(x) = W,(—2), (31c) The initial values will be taken as s(0) = s, = +1, 
Wx) = Wi + Ws + We, x(0) = y(0) = 0. If s(0) = —1, it is clear from symmetry 
: ‘i és that W,, W, are to be interchanged, and (z, y) replaced 
= 20 ea) a"): (82) by (—2z, —y). For arbitrary initial values (2, yo), (a, y) 

| eis ; 1 laced by ( — x — 2% t — Yo). 

Th 1s rep yi oY 0 Yo 

e normalizing constant C is Let w(z, y, ), q = 0, 1, 2, --- , be the probability 
a T[(u/2)(1 — w + 28)ITI + u(1 + a))/2] ‘ density of (x, y) at time ¢ together with the probability 
4 re (33) that exactly q zeros of input occur in (0, f), conditional 


2 Var TMw)T (us) 


| The densities simplify for special values of the param- 
sters. For example, if 1 + ul + a — 28) = —2K, 
K = 0, 1, 2, --- , the series (29), (30) stop after K + 1 
erms. In this case, the total output density W(x) can 
de written in terms of a Jacobi polynomial’’ of Kth degree: 


K! TQu + 2K)Tus — K) 
FO“ TUB) + KE 


mee Pe Oe? — 1), |al<1. 34) 


(t is interesting to note that if K = 0, 7.e., 26 = w* + 
| + a, W(x) is identical with the output density obtained 
vhen the input is a Rice telegraph signal.’® The three- 
state signal actually reduces to the latter only when a — 1. 
_ Another class of simpler distributions is obtained when 
: is small, and also 26 = uw’ — (1 — a). Then 


Tid + w + ua)/2] 
Pw)Tid — w+ ua)/2] 


.| x (ea es LGA 


Wa) = 


V(x) = 


(35) 


The moments of W(x) are readily obtained from (32), 
since integration leads to a 2F, series with argument 
mnity, which can be summed. It can then be verified 
hat for arbitrarily-fixed a, 6, and large u, the distribution 
of aVY( + wp + wa)/2p, is asymptotically Gaussian with 


7ero Mean and unit variance. 


Hzample 2: Doubly Integrated Rice Telegraph Signal 


Let {s(¢)} be the Rice telegraph signal with s, = +1, 
» = —1, and a, y the output of the first and second 
ntegrator, respectively. Eqs. (6) and (8) for this case are 


(d/di)\(z, y) = (8, x); (36) 
(x, y) = Fit; Xo, Yo) 
= (a + 8,t, ot + yo + 8,t'/2). (37) 
Zqs. (13) become the parabolic system 
dW, /dt + OW,/dx + xdW,/dy + wW, = wW, (38) 


OW./dt a OW./dx + xoW,/dy + uW, uW,. 


165 A. Erdélyi, et al., “Higher Transcendental Functions,” 
AicGraw-Hill Book Co. Inc., New York, N. Y., article 10.8.; 1953. 

16 W. M. Wonham, and A. T. Fuller, “Probability densities of 
he smoothed random telegraph signal,” J. Electronics and Control, 
ol. 4, pp. 567-576; 1958. [See (27).] 


on the initial values. Since g = 0 with probability e~“’, 
w(x, y, ) =e de — t) ty — #/2). (89) 


If one input zero occurs in (0, 4), and s(0) = —1, the 
locus of terminal points at time ¢ is found from (37) to 
be 


y = vila, t) = [(¢ + 2) — 20)/4, [eral <onte (40) 
Similarly, if s(0) = +1, the new locus is 
y= vr, t) = (= — 2) 4271/4, ale: (41) 


Using (87), (40), and (41), it can be shown by induction 
on the number of input zeros in (0, ¢) that the domain 
of the W; is the conoid 


Civ(z,) Sy < ya, b), 


The characteristic surfaces y = wy; are shown in Fig. 1. 


[oer atenO eC) 


at) 


Fig. 1—Domain of probability densities W,(x, y, t) for a doubly- 
integrated Rice telegraph signal s(¢) = +1. 
dx/dt = s(t); dy/dt = x(t) 
Iz] < t; Yilz,t) Sy S yale, 2). 


(+), (—) denote trajectories corresponding to s(t) = +1, —1. 


To find boundary values of the W; on these surfaces 
it is sufficient to compute the first few functions w (2, y, é). 
By an argument similar to the one which led to (9), one 
can write 


Ti 


way) =f etwMP(R(—7, 2, 05 t— de, 
0 

(43) 
Since s(0) = s,, 7 = 1 or 2 in the integrand, depending 


g=1,2,::: ~5 
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on whether q is even or odd. The integration is taken over 
all previous instants ¢ — 7 at which the (q — 1)st zero 
may have occurred. The upper limit 7; is therefore defined 
by the intersection of the Pa ea ory ayy = Fp(— 734,59) 
with the boundary surface y’ = y.(v’,¢ — 7) Gf7 = 1), 

Ory = Ve, § or) Gt 7 = 2) From (37), (40) and (41) 
there results 


n= [vale, ) — vl 2), ey 
t2 = [y — valx, )\/(t + 2). 
From (39), (43), and (44) one may now derive 
w(x, y, t) = (ue *'/2) dy — ys), (45a) 
w(x, y, t) = we" /2(t — 2), (45b) 
w(x, y, 2) = (ure “*/4) log ee — al (45c) 
wert, de=Os ag slg? | ge 4b oe oO) 


Since s(t) = s, or 8,, depending on whether q is even or odd, 
the space densities W; are given by 


fo) 


We= dw ae 
sg (47) 
W pes — (2a+1) f 
2 » W 
Thus the boundary values are, finally, 
a (2) i 
Wiz, V;, t) = Ww (es V;; t), (48) 


W.(a, Y, t) i w(x, Yy; t) as y ae Y;. 


Eqs. (38) and (48) have been solved up to a Laplace 
transform with respect to y. On setting 


Yo 
W(x, P, t) 7a i Cte Wa; Y; t) dy, 


1 


(38) and (48) become 
aw, /at Sf AW, /dx patel px) W, 


— (u’/2) exp (—ut — py) = wW, 
OW,/dt — OW./dx + (u + px)W. = uW,, (49) 
oe ies exp (—ut — pl'/2), x= 1 
0 ¢= -1 
W(x, p, }) = 0, x= tt. (50) 


The transformed system is hyperbolic, with boundary 
values given on the characteristics x = +t, t > 0, and 
can be solved by standard methods. The results may 
be written in terms of confluent hypergeometric functions: 


Walz, p, t) = (w’/4)(t + 2) exp (—ut — py) 


-(F, [1 + u°/2p; 2; p(? — 2”)/2]} (51) 
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‘W.(a, p, t) = (u/2) exp (—ut — py) 


Ely a} = 


Decemb: 


Fle /2p31; p@ 2/21 


If s(0) = —1, the new transformed space densitie 
are obtained by interchanging W,, W,. and replacin 


(x, p) by (—2, —p). The total space density condition: 
on 
x(0) = y(0) = 0 5 
s(0) = +1 or —1 _ with equal probability 
is, therefore, 
We, P; t) = 4(W,(c, P; t) os W.(—z, = Ps t) 


oF W(x, P, t) = W.(-2, er a oie t)] 

= (u/4e"' "2 {2 PF, [u’/2p; 1; p(? — 2°)/2] 

+ [ut + p(t? — 2*)/2] .F.[l + o°/2p; 2; pt — 2 \/a 

a gene py, (6 
In (54), Kummer’s transformation’’ was used to evaluat 
W; (ae —P, t). 

The marginal density for x(t) alone can be found o 
integrating w°? (+a, +y, t), w (4a, +y, b) with respec 
to y and setting p = 0 in (54); this result has been di 


cussed by McFadden."* The expectation of y(t) condition: 
on x«(#) = x and initial values (53) is simply 


= ry 


Ety | x} 


The conditional variance of y(t) can be obtained fror 
(45a), (55), and the coefficient of p’/2 in (54). The exac 
result is somewhat involved; however, two approximatior 
are 


[E{y | x}]° 


(f — 2°)? ut 
See 1) SE out) as te | 
e 16 ( ut) (5 


2 _ 3/2 
ae + 0(4 :); t) oe fimed® “jp => te" 
12u 


If | x(t) | < # and ut — 0, only the boundary densitie 
w'? (+2, +y, 4) contribute to the conditional variance 
For wt > 0 and small the only first-order term arise 
from w” (+a, ty, #). When yu is large, the absolu; 
variances of w(t) and y(t) are.both small, and the co 
ditional y-distribution sharply peaked at the mean. 
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) Summary—A computationally convenient method is presented 
or simulating a sequence of sampled values of a stationary Gaussian 
wrocess having a specified correlation function or power spectrum. 
7-transform theory is applied to provide a simple recursive formula 
or generating the values from a set of independent Gaussian 
‘andom variables. 


INTRODUCTION 


HE PROBLEM considered here is that of gener- 
ating numerically from a table of independent Gaus- 
sian deviates’ a finite sequence of random variables 
waving the statistical properties of a set of uniformly 
spaced samples of a stationary Gaussian process with a 
‘pecified correlation function or power spectrum. Such 
sequences are useful for the digital simulation of noise 
a and for other purposes. Stein and Storer’ have 
discussed some applications of these sequences and also 
considered several methods for generating them. Two ap- 
sroximate methods were suggested but the effects of the 
oe were not stated. In addition, an exact 


rocedure was presented which, for the computation of NV 
sample values, requires the computation of the eigen- 
rectors and eigenvalues of the covariance matrix of the 
sequence, which is of order N. Marsaglia’ pointed out 
hat equivalent results can be obtained by the much 
simpler procedure of factoring the covariance matrix into 
‘wo triangular matrices and then multiplying the sequence 
of independent samples by one of these matrices. However, 
‘or large N, say 100, this is still a formidable compu- 
;ational problem. This paper provides a greatly simplified 
yrocedure for accomplishing this task when the desired 
sequence is derived from a stationary process with known 
‘orrelation function or power spectrum. In this case the 
1ecessity for factoring the covariance matrix is eliminated 
ind after some relatively short preliminary calculations 
he sample sequence is given by a simple linear recursive 
ormula which is applicable for N arbitrarily large. 

It is assumed that the process which is sampled has 
nean zero and a power spectrum P(w’) which is rational 
ind of order K in w’. The method described here requires 
ither the correlation function or the power spectrum of 
he sampled time series. For simplicity we assume that 
his sampled time series is rescaled in time so that its 


* Received by the PGIT, February 15, 1960. ; 

+ RCA Missile Electronics and Controls Div., Burlington, Mass. 

1M. E. Muller, ‘A comparison of methods for generating normal 
leviates on digital computers,” J. Assoc. for Computing Mach., 
rol. 6, pp. 376-383; July, 1959. ; , 

29. Stein and J. E. Storer, “Generating a Gaussian sample,”’ 
RE Trans. on INrorMATION THEORY, vol. IT-2, pp. 87-90; June, 
a. Marsaglia, ‘A note on the construction of a multivariate 
1ormal sample,” IRE Trans. on Inrormation Tueory, vol, 
‘T-3, p. 149; June, 1957. 
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Generation of a Sampled Gaussian Time Series Having a 
Specified Correlation Function* 


MORRIS J. LEVIN}, memser, IRE 


sampling interval is unity. Then if the correlation func- 
tion p(r) of the continuous process is given, the cor- 
responding rescaled sampled correlation function is 


DOR) T= DUCL a) Orca lpeeeecpecen: (1) 


where T is the sampling interval. If the power spectrum 
P(w") is given, then the corresponding sampled power 
spectrum can be determined by the method described by 
Ragazzini and Franklin.* 


NOTATION 
The following notation is employed: 


B= Eg 


o(m) 


I 


the correlation function of the sampled time 
series to be simulated. 


the sampled power spectrum 
corresponding to ¢(m), 1.e., 
the z transform of ¢(m). 


20) = D> ome” = 

u(n), n > 0 = a sequence of independent Gaussian 
deviates with mean zero and variance one. 

v(z),0 <i < K — 1 = an auxiliary set of independent 
Gaussian deviates with mean zero and variance 
one. 


y(n), n > 0 = the random sequence having correlation 
function ¢(m) which is generated from u(n). 

—,,0 <2< K —1 = an auxiliary sequence of random 
variables which is generated from v(2). 

R;; = covariance of €; and &;. 


Gra ee Toe aoe 


1+ bet + eee H+ be ™ 

the pulse transfer function of a stable linear 
sampled-data filter the output of which is a 
time series having correlation function ¢(m) 
when the input is a sequence of independent 
random variables with variance one. 


(bk #0) = 


h(n) = the impulse response of the filter H(z). 
I’ = the unit circle in the complex plane. 


E = statistical expectation (mean) operator. 


DESCRIPTION OF THE PROCEDURE 


The basic idea of the procedure described here is to 
calculate the transfer function H(z) of a linear filter which 


4J. R. Ragazzini and G. F. Franklin, ‘“Sampled-Data Control 
Systems,’’ McGraw-Hill Book Co., Inc., New York, N. Y. p. 259; 
1958. 
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would convert white noise into noise with the specified 
correlation function ¢(m), and then to use H(z) expressed 
as a recursion relationship to compute y(n) from u(n). 
For the filter H(z) to generate a stationary random out- 
put, it must be operating in the “steady state;’’’’® 7.e., 
it must have had an input of white noise for alln > —o. 
However, for computational purposes the input sequence 
must begin at » = 0. The procedure described here 
provides the correct “‘initial conditions” by generating K 
values of y(n), 0 <n < K — 1, by a special method so 
that these AK values together with the corresponding 
values of u(n) have the same covariance matrix as if 
the filter were operating in the steady state. This is done 
by replacing the effect of the input sequence for n < 0 
by K auxiliary random variables, £;, which are generated 
from the auxiliary independent random variables v(7). 
The simplicity of the approach rests on the fact that in 
a practical problem K will seldom exceed two or three. 
Then the remaining values of y(n), n > K, can be com- 
puted recursively. The steps in this procedure are des- 
cribed below. 


Step 1: Determination of H(z) 


If S(z) is not specified, it must be calculated by taking 
the z transform of ¢(m). Then H(z) is determined by’ 


(2) 


Since H(z) is assumed to be stable it has all its poles 
within T and H(z’) has all its poles outside. Therefore 
H(z) is found by factoring @(z) and associating appropriate 
poles and zeros. Since the zeros of H(z) are not restricted 
to be within IT, there are alternative forms for H(e), 
depending upon which zeros of ®(z) are associated with 
H(z) and which with H(z~’). 


&(z) = H(z)H(e""). 


Step 2: Determination of h(n) 


In general, 


H@) = > hinje”. (8) 


h(n) is easily obtained by expanding H(z) by long division. 
Only the first K values of h(n) are required here. 


Step 3: Determination of y(n),O <n < K-11 


A. Formulation: The output of a linear sampled-data 
filter is a linear function of the present and all previous 


° B. Friedland, ‘Least squares filtering and prediction of non- 
stationary sampled data,’’ Information and Control, vol. 4, pp. 
297-313; December, 1958. 

6 D. G. Lampard, ‘“The response of linear networks to suddenly 
applied random noise,’ IRE Trans. on Crrcurr Turory, vol. 
CT-2, pp. 49-57; March, 1955. 

7R. H. Barker, ‘The pulse transfer function and its application 
to sampling servo systems,’”’ Proc, IEE, vol. 99, pt. 4, Monograph 
43; July 15, 1952. 
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‘input values, 
y(n) = x h(m)a(n — m) ( 


where x(n) represents the input to the filter. Then we ca 
write, for example, 


y(0) = D2 Km)a(—m) = KO) +f 
where 
ee »D h(m)a(—m). ( 


In general, we can take 


y(n) = = Aiman—m +, n<K-1 — €¢ 


where 
, = DB h(m + n)a(—m) (i 


represents the influence of all x(n) for n-< 0. The virtu 
of this representation is that if the correlation functio 
of x(n) is known, then the covariance matrix of the . 
can be determined. With this matrix available, values « 
the £; can be generated having the appropriate statistic: 
properties so as to simulate the effect of all x(n) for n < | 

B. Covariance Matrix of the €;: For the situation cor 
sidered here, the x(n) are statistically independent so tha 
for example, 


Roo = Varig, = al » hime(—m) | 


I 


> ne(m) — n°(0) j 


$(0) — h°(0). 


From z-transform theory,’ ¢(0) can be determined fro 
@(z) by taking 


$(0) = >> residues of poles within I of ve (1 


In general, we find 


Ri; = cov &;, &;) 


© o-)} 


EX) Shim + Dh(p + ja(—m)x(—p) 


m=1 p=1 


I 


I 


X hm + Dm + 9 a 


Il 


> ADA + Vd) 


k=i+1 


Il 


6G —) — DY Wmnlm + j = 9 


1960 


and 


¢(m) = >> residues within I of [e'"'-? B(z)]. (12) 

C. Determination of the £;: After having calculated the 
covariance matrix of the K random variables £;, appro- 
priate sample values for these variables can be obtained 
by linearly transforming K auxiliary independent Gaussian 
deviates, v(z). A convenient method for doing this takes 
the form 


) cs OE (13) 
i=0 

The values of the c;; can be found by the method des- 
eribed by Marsaglia.® For convenience we list here 
recursive formulas for the first few c;;, which are sufficient 
mK < 3. 


1 


S00 = Roo , 

) 

: Tea ae 

fio = ee a tok = SU sige =~ Git 

00 

(14) 
: 

! 20 Roi = Cr0C20 2 om 
C29 = C2 lea » C2 = V Ro» = Gx = ERS 
| 00 Ci 


Note that only these K values of the £; are required no 
matter how large N is. 


Step 4: Determination of y(n) forn > K 


The transfer function 


eu omc? -K 
Bom sap ee 
is equivalent to the recursive relation 
vln) a Ut) 1) —* * — deyn — K) + agu(n) 
: + aun —1) +--+ + aun — K). (16) 


: 
c 


_ The procedure described in the previous sections 
grovides K consecutive pairs of corresponding values of 
y(n) and u(n) having the same statistical properties as 
f they were produced by steady-state operation of a 
ilter having the transfer function H(z). Therefore, any 
Jesired additional number of values of y(n) can be gene- 
ated by applying the recursive relation (16) starting 
with these initial values. We have, in a sense, provided 
the correct initial conditions, so that the filter now behaves 
us if the sequence u(m) had been applied to it for all 
2 > —o and not just n > 0. It should be noted that 
she y(n) sequence which is generated is Gaussian since 
avery value is obtained as a linear combination of Gaus- 
sian variables. 
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Example 1: 


Given - 


SCalteal) 


(Dag) at 3 
then from (1), 
om) = eer 


The z transform of ¢(m) is obtained by taking the 
sum of the individual z transforms of the parts form > 0 
and m < 0. Letting A = e°°” we have 


i) 1 
gy bates 


1 


MISA ae 
je ee ee 


oO) fe 1 Az 


From (2) and (8) 


BVA oe 


soles 1 — Ag” 


l| 


A/ Vee A? [ede ee eee 


I 


Then 
h(0) = V1 — A’, 
so from (9) and (14) 
Co = Ve0) — WO) = A, 
and from (13) 
Eo) = Coo 0(0) = Av (0). 
Then by (5) 
(0) = V1 — A? ud) + Av(0). 


Since u(0) and v(0) are independent and their values do 
not enter the expression for y(n) for n > 1, y(Q) can be 
generated more simply from a single random variable 
having the appropriate variance, by taking 


yO) = (V1 — A*)’ + A*PPuO) = uO). 
Finally for n > 1, from (16) 


y(n) = V1 — A? un) + Ay(n — 1). 


Example 2: 
Given 


642” : 
82* + 5427 + 1012? + 54e¢ +8 


P(z) = 


By factoring we find 


H(z) = = iI. = = =3° 
(e+ 4(e+3)  1+4+.0.752° + 0.1252 
By long division h(0) = 1 and A(1) = —0.75. Referring 
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to (10) and (12) 
¢(0) = >> residues within T' of Eg 
= 64/35 
¢1) = DS residues within I’ of &(z) 
= —128/105. 
Therefore from (11) 
29 —197 _ 149 
Roo = 35 ) Ro Fi ~ 420 ) Ri, 560 
Substituting in (14), 
Coo = 0.910, (Sry) —0.515, Oe 0.023, 
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and from (7), (13), and (16), 


y(0) = u(O) + 0.91 (0), 
y(1) = u(1) — 0.75 u(0) — 0.515 v0) + 0.028 v(1), 
y(n) = —0.75 y(n — 1) — 0.125 y(n — 2) + un) n2>q 


CoNCLUSIONS 


The procedure presented here provides an exact an 
computationally practical method for solving the probler 
considered. It might appear that the sequence of step 
described for obtaining y(n) for 0 < » < K — 1 coul 
be consolidated into explicit formulas, but no method ¢ 
doing this, which results in any simplification, has bee 
found. 


A Note on the Local Structure of Shot Noise* 


RICHARD A. SILVERMAN}, SENIOR MEMBER, IRE 


Summary—It is shown how to construct shot noise which, like 
the turbulent velocity field, is quite accurately univariate normal, 
but exhibits marked departures from bivariate normality at close 
ranges. 


T is well known that the random velocity field pro- 
] duced by wind tunnel turbulence at high Reynolds 
numbers has the following statistical structure: Let 
the x axis be directed downstream beyond the grid gene- 
rating the turbulence and let u, (x, y, z) be the downstream 
component of the random velocity field, measured at any 
point of the flow. Then, to a good approximation, wu, (2, y, 2) 
is a normal random variable, z.e., the velocity field is 
quite accurately univariate normal. If the field were bivar- 
iate normal as well, then writing 6u(r) = w.(a + 7, y, 2) — 
u(x, y, 2), where again (a, y, 2) is any point of the flow, 
we would expect the random variable 6u(r) to be normal, 
in particular to have a skewness 


y(r) = Eléu(r)]}*/{ H[ou(r)P}°? 
equal to zero and a flatness 
o(r) = Eldu(r)]*/{H[ou(r) P} 


equal to three, the appropriate values for a normal random 
variable.’ What is actually found to be the case instead 
is that y(r) and ¢(r) are appreciably different from zero 
and three, respectively, for values of r such that the cor- 


* Received by the PGIT, April 15, 1960. The research reported 
in this paper has been supported by the AF Cambridge Res. Center, 
Air Res. and Dev. Command, under Contract No. AF19(604)5238. 

+ Inst. of Mathematical Sciences, New York University in New 
York. 

1The symbol # denotes the expectation value or ensemble 
average. 


relation function 
f(r) a Buy (« = Tr, Y, z)U, (2, Yy, z)/Elui (a, y, z)]’ 


is appreciably different from zero. For experimental value 
of y(r) and ¢(r) and a detailed discussion of the statistiea 
structure of turbulence, we refer to the last chapter a 
Batchelor’s monograph.” 

The state of affairs just described raises the followin 
question: Is it possible by suitably choosing the sho 
rate and the pulse shape to construct shot noise whic. 
mimics the statistical structure of turbulence, 7.e., whic 
is quite accurately univariate normal but exhibits marke: 
departures from bivariate normality at close ranges? T 
answer is in the affirmative, as we shall now show. Fa 
simplicity, we consider first the case of a one-dimensioné 
random process 


=f bet-w-0Ve 


where the ¢; are the random occurrence times of the even 
in a stationary Poisson process with an average rate ¢ 
p events per second. The @; are a family of independer 
random variables, all uniformly distributed over the uni 
interval (0, 1). The centering constant c is chosen to I 
V3/2 J%. s(t) dt, which assures that EKu@) = 0, a 
the normalization is such that Hu’(t) is independent of 
and is in fact equal to [®., s*(é) dt. (A similar constructic 
has been given elsewhere.*) 


* G. K. Batchelor, ““The Theory of Homogeneous Turbulence 
Cambridge University Press, Cambridge, Eng.; 1953. | 

° R. A. Silverman, ‘‘An isospectral family of random processe 
IRE Trans. on Inrormation Tuxory, vol. IT-6, pp. 485-4 
September, 1960. 
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We must now suitably choose the pulse shape s(t) and 
the rate p so that the random variable u(t) is quite ac- 
curately normal, while for small 7 the difference random 
variable u(t + +r) — w(t), in particular w’(t) = du(t)/dt 
corresponding to the limit 7 — 0, is quite markedly non- 
normal.* To achieve this, we choose s(t) to be the function 


t/2e, O< t < 2¢, 
: 1; DAIS, = 
| Sie Cet are €, (2) 
@=D/e “a—«<t <a, 
0, a<t, 
with derivative 
1/2, Osteo De: 
cay = 2<t<a-e, 
—l/e, a-—-ex<xt<a, 
0, a <t, 


where « < a is a small parameter to be adjusted later. 
Our choice of s(t) is motivated by the fact that its sup- 
sort (a) is much larger than the support (3¢) of its de- 
tivative; the significance of this will emerge presently. 
We have also arranged to give u(t) a negative skewness, 
n order to resemble the turbulent velocity field. 

_ We now use the fact that the semi-invariants u, of 
4(t) are given by the formula 


if n/2 © sg 
we A2) Eg" cc 8) at, 


where @ is any of the 8;.° In our case Hp" = 1/(n + 1), 
ince 8 is uniformly distributed over the interval (0, 1); 
she inclusion of the random variables 6; in (1) is to assure 
shat the distributions of u(t) and w’(t) contain no delta 
‘unction terms. In view of the relations p2 = M2, us = Ms 
ind 4 = mM, — 3m; between the semi-invariants pu, ps, 
ind yw, and the central moments mz, m3, and m,, we find 


hat the skewness of u(é) is given by 
f 
M3 

CV aie bees 72 
Me 
ind its flatness by 

fae ei leit 
g “+ i 
or the case of the pulse shape (2), we find that 


: cies 
3 & pet ela 
— 4 p (a = Daye \/ pee ? 


eX a, 


_ 4 The parameter ¢ is fixed but arbitrary. ii 5 

5S. O. Rice, ‘Mathematical analysis of random noise,” re- 
rinted in the collection “Selected Papers on Noise and Stochastic 
’rocesses,” N. Wax, ed., Dover Publications, Inc., New York, 


I. Y., pp. 150-157; 1954. 
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and 
(o4 = € 
PE. 5 ieee 
Co) RES AER OF un See” if e Xa, 


so that for small ¢«/a, the condition Wpa > 1 assures 
that u(t) is quite accurately normal.° On the other hand, 
differentiating (1), we see that the univariate distribution 
of the process u’(t) is governed by the semi-invariants 


3 n/2 cy 
ws (2) up" [ (sor at, 
with corresponding skewness 


Fae ee coreg 
4 /2p¢ V pe’ 


and flatness 


0.9, 
pe 


I 


a 
Examining the expressions for y, ¢, y’, and ¢’, we see 
that by satisfying the conditions Vpa >> 1 and V pe ~ i 
we can arrange to have simultaneously a quite accurately 
normal distribution of u(t) and a markedly non-normal 
distribution of w«’(t). Moreover, the two conditions 
V/ pa > 1 and Vpe ~ 1 are compatible, provided only 
that Va/e > 1. For example, if a = 1, « = 10°* and 
p = 10°, we have y ~ 0,¢ ~ 3, y’ ~ —0.5, and ¢’ ~ 3.9. 

The skewness y(7) and the flatness ¢(r7) of the difference 
u(t + r) — u(t) can be calculated in just the same way 
by observing (following a suggestion of Gilbert) that 
the process u(f + 7) — w(t) is given by (1) if we re- 
place s(t) by s(t + 7) — s(t). It is found that as r — 0, 
y(r) and ¢(7) reduce continuously to the limiting values 
y’ and ¢’, and that as r approaches the correlation distance 
a of the process u(t), y(7) and ¢(r) approach the normal 
values of zero and three.’ Thus, w(t) resembles the turbu- 
lent velocity field by having a markedly non-normal 
bivariate distribution at close ranges. 

Precisely the same kind of construction can be carried 
out in three dimensions by replacing the random times 
t; by a spatial Poisson distribution and replacing the 
pulses s(t) by three-dimensional ‘‘blobs.” For example, 
suppose that 


u(x, Y, 2) 


= Je me B s(a a x sly ee yisZ ie z;) me’ V py; 


6 See the remarks in D. Middleton, ‘“‘An Introduction to Sta- 
tistical Communication Theory,’ McGraw-Hill Book Co., Ine., 
New York, N. Y., pp. 505-506; 1960. 

7 The correlation function of u(t) is 


iy . s(t + 7)s(t) dt of: it ' s(t) dt, 


which vanishes for +r > a. The function y(7) undergoes a sign 


change in the interval (0, «). 
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where the function s is the same as in (2), while this 
time, p, is the volume density of the points (a, yi, 2:) 
of a homogeneous spatial Poisson process and the sum- 
mation is over all the ‘‘centers” (x;, y;, 2;). For each 
point (2;, yi, 2:), 8; 18 an independent random variable, 
uniformly distributed over the interval (0, 1). The center- 
ing constant c is now 


a { i: ; s(z) aw), 


and 


Hu@;sy,2)= 0, EWG@;yi2) = if s(x) any 


For the skewness y and the flatness @ of the random 
variable u(x, y, 2), we now have 


1.8 
ad ’ ~3 Sear Sia 
V Par g oi poke 


while for the skewness y’ and flatness ¢’ of the random 
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Variable (0/dx) u(x, y, 2), we find® 


She ee ee 
i 4 /2p,a’e ~AN/ pyat e 
7 0.9 

Nps annie 


If V p,o8 > Land Vp,a’e~ 1, the random field u(x, y, 2 
is quite accurately univariate normal but exhibits markec 
departures from bivariate normality at close ranges 
These two conditions are compatible, provided tha 
Va/e >> 1, as before. For example, if a = 1, « = 10° 
and p, = 10*, we have y ~ 0, ¢ ~ 3, vy’ ~ —0.5 anc 
g ~ 3.9. 

In conclusion, we see that when a large number 0. 
elementary waveforms are superimposed, even with higl 
density and considerable overlap, there is no reason t 
expect a priori, in the absence of detailed informatior 
about the shape of the waveforms, that the resulting 
process is accurately normal. 


8 We use the obvious modifications of the formulas for the semi 
invariants uw» and pn’. 


Maximum-Weight Group Codes for the 
Balanced M-Ary Channel" 


CARL W. HELSTROM{ 


Summary—tThe construction of (n, k) group alphabets is discussed 
for the balanced M-ary channel, where M is the power of a prime. 
In this channel all M digits are equally likely to be in error, and 
an incorrect digit is equally likely to be any digit besides the one 
sent. The alphabets are formed by taking n columns of the modular 
representation table of the Abelian group of k-tuples of elements 
from the Galois field GF(M) under digitwise addition. The forma- 
tion and properties of that table are described. Attention is focused 
on alphabets in which all letters except the n-tuple of 0’s have 
the maximum number of non-null elements. Tables of such alpha- 
bets are given for M = 2, k = 2, 3, 45 M = 3, k = 2, 3; 
and M = 4,k = 2,3. 


I. Toe Batancep M-Ary CHANNEL 


N the M-ary channel there are M different signals 
available, each corresponding to one of a set of M 
digits. One signal is sent at a time, say every T 


* Received by the PGIT, March 17, 1960; revised manuscript 
received, May 19, 1960. 
+ Westinghouse Res. Labs., Pittsburgh, Pa. 


seconds. The channel is termed ‘‘balanced,”’ if all digi , 
are correctly received with the same probability, an 
an incorrect digit is equally likely to be any digit othe 
than the one transmitted. 

An example of a balanced M-ary channel is one wit! 
M orthogonal signals f;(¢) of duration T, 1 <j < M 


ih flif,(t) dt = 0 


for 7 ¥ j, All signals are received with the same energy u 
in white Gaussian noise of bandwidth much larger tha: 
1/T. The receiver contains M filters in parallel, eac 
matched to one of the orthogonal signals. If the signa’ 
are narrow-band modulations of a carrier with randop 
phase, the matched filters are followed by detector 
whose outputs are measured at the end of each recepti 
interval of duration 7. The receiver emits that dig) 
corresponding to the filter whose detected output _ 
largest. Reiger has given a formula for the probability 


960 


hat a digit is correctly received:* 


a= D0 (™) ex [-@ — nEpm, 


vhere N is the spectral density of the noise. A frequency- 
hift-keying system with M well-separated frequencies 
an be represented as a balanced M-ary channel. 

_ Error-correction codes suitable for such a channel have 
yeen proposed by several writers, mostly for special 
ralues of M.*~° The codes studied by Lee® and in most 
f Ulrich’s paper” are designed for a different kind of 
:onbinary channel, one whose M digits are somehow 
ordered, with the noise more likely to change a digit 
nto its nearest neighbors than into other, more distant 
ligits. Some properties of single-error correcting codes 
or the balanced M-ary channel, M the power of a prime, 

ve described in another paper.’ 


II. Group ALPHABETS FOR THE 
BaLancep M-Ary CHANNEL 


| The letters of a group alphabet can be considered as 
ements of an Abelian group.* In the group alphabets 
lescribed here, the letters are n-tuples of the M-ary 
ligits of the balanced channel, and M is the power of 
prime. The total number of such n-tuples is M*, L = M* 
if which are selected as code letters. We call such an 
Aphabet an “(M, n, k) alphabet.” Many of Slepian’s 
esults for binary (n, k) group alphabets can be applied 
vith obvious changes to these (M, n, k) alphabets. 

Each digit is assigned to an element of the Galois 
ield of order M, abbreviated GF(M).°’” The alphabet is 
hen a subgroup of order M” of the group C’ of all M"n- 
ples, in which the group operation is digitwise addition, 
erformed by the rules for the Galois field.” We denote 
ry O the unit element under addition and the digit assigned 
o it, and by 1 the unit element under multiplication and 
ts assigned digit. The remaining M — 2 digits are called 
‘nonunit” digits, and all digits except 0 are called “‘non- 


br2'S. Reiger, ‘‘Error rates in data transmission,’ Proc. IRE, vol. 
6, p. 919; May, 1958. 
2W. Ulrich, “Nonbinary error correction codes,” Bell Sys. Tech. 


"., vol. 36, pp. 1341-1388; November, 1957. See Sec. IV, pp. 1360- 
367. 


| *M. J. E. Golay, ‘Notes on the penny-weighing problem, loss- 
ass ae coding with nonprimes, etc.,’ IRE Trans. on Inror- 
{ATION THEORY, vol. IT-4, pp. 103-109; ‘September, 1958. 

fee. Ss. Shapiro and D. L. Slotnick, “On the mathematical 
heory of error-correcting codes,” [BM "J. Res. Dev., vol. 3, pp. 
‘5-34; January, 1959. 

5 J. Cocke, “Lossless symbol coding with nonprimes,” IRE 
*RANS. ON INFORMATION Tueory, vol. IT-5, pp. 33-34; March, 
959. 
6C, Y. Lee, “Some properties of nonbinary error-correcting 
odes,” IRE Trans. oN INForRMATION TurorRy, vol. IT-4, pp. 
7-82: June, 1958. 

70, W. Helstrom, “Single-error correcting codes for nonbinary 
alanced channels,” TRE TRANSACTIONS ON INFORMATION TuHEorRy, 
o be published. 
8D. Slepian, “A class of binary signaling alphabets,” Bell Sys. 
"ech. J., vol. 35, pp. 203-234; January, 1956. 
9B. M. Dwork and R. M. Heller, ‘Results of a geometric ap- 
roach to the theory and construction of nonbinary, multiple error 
nd failure correcting codes,’ 1959 IRE NatronaL CONVENTION 

ORD, pt. 4, 123-129. 
B. i, van GER Waerden, ‘‘Modern Algebra,”’ F. Ungar Publish- 
rg Co., New York, N. Y., Sec. 37, pp. 115-119; 1949. 
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null.”” We define the “weight”’ of an n-tuple as the number 
of non-null digits it contains. All alphabets include the 
letter J = (000 --- 0) of weight w, = 0. 

These group alphabets are well suited to the balanced 
M-ary channel when messages are coded so that all 
letters are sent equally often. The received sets of n 
digits are to be decoded by assigning to each the code 
letter with the largest posterior probability of having 
been sent. This can be done by dividing the group C™ 
into cosets, using as the leader of each coset the element 
with the smallest weight, and subtracting’? from the 
digits of each received set those of the leader of its coset. 

One can think of the received n-tuple as the digitwise 
sum of the transmitted letter and an n-tuple caused by 
the noise. In order for a letter to be correctly received, 
the noise n-tuple must be either the letter J = (000 - - - 0) 
or one of the (M** — 1) coset leaders. Therefore, the 
probability Q of correctly receiving a code letter is 


Q Q” ae 2 Ga pee 


p= (Ll —/M 1), 
P< Mone ol, 


I 


where w(z) is the weight of the 7th coset leader. 


III. Tat Moputar REPRESENTATION TABLE 


Like binary group alphabets,® these (IZ, n, k) alphabets 
can be conveniently constructed by selecting columns from 
the modular representation table (MRT) of the Abelian 
group C}’ of order M“, which is formed by all k-tuples 
of elements of GF(M) under digitwise field addition. This 
MRT contains M* rows and columns, each labeled with 
a k-tuple of field elements from GF(M). At the inter- 
section of the row labeled (b,, b., --- b,) and the column 
labeled (a,, @2, «++ a, 1s found the field element 


a,b; + azb2 + 


in which the addition and multiplication are performed 
according to the rules for the Galois field. The MRT 
for M = 2,k = 4 is given by Slepian.*® 

The columns of the MRT form a representation of 
the group C7’ in the following sense. If the elements of 
any two columns are added row-by-row by the rules 
for the field, a new column results that is the same as the 
one labeled with the group product (digitwise sum) of 
the labels of the first two columns, because the field 
operations obey the distributive laws.’ Henceforth, we 
omit the column and row of all ‘0’s in the MRT. 

The remaining columns and rows of the MRT fall 
into sets of (M — 1) columns (or rows) that differ only 
by a multiplicative factor. By selecting one column and 
row of each set, we can write down a ‘“‘reduced MRT” 
(RMRT) that has only 


K = (M* — 1)/(M — 1) 


: and; 


u “Subtraction” is the operation inverse to Galois field addition. 
12 yan der Waerden, op. cit., p. 32 


552 


rows and columns; it exhibits well enough the structure 
of the original MRT. (For M = 2, the RMRT is the 
same as the MRT.) In Table I, we give an RMRT 
for M = 4,k = 3; it isa 21 X 21 matrix. The field ele- 
ments are now 0, 1, a, and @; they obey the rules” 


ge = 0, (x any element) (1) 


lt+e+B=0, Co bon — I 


B = a-a, 


TABLE I 
RepucED MopuLar REPRESENTATION TABLE OF C3! 


a) 
Fe  ) 
OOo OCOCrF 
a a) 
a) 
ord BFS 
ore 
Ror 
RFHOoD BOF 
ee © ore 
Fk Oo CORE 
KDOo ODre 
eee 
QR te 
— TD ae 
8k = 
ae 
De 
KDD Boe 


Re 


001 
010 
100 


—SRDm BWRE 


oo = 
on 
me OR 
— ee 
sae) 
eRe 
ek RQ 
KDoe 
EDR 


O11 
Ola 
018 
101 
10a 
108 
110 
lad 
180 


lll etl sell atl aot 
RRR DBORR ED 
R@DBOR RR Ree 


COCR RRB@BD!] CRBHeHKHKRAR 
DOR BPRHDBDRD 
RWORBWOR ER OS 


SCRBORBBWeEO 


Re DBODBRFR OC CRB@WBOR COBH 
SRR @BR REPROD! DOR RBWOOCeER 
ROR RDRDHO | WOR OR BR OF 


EPR OR KFRBOBeH 


DRE OOCORFE 
DMR HR RHBBDOR 
RRR DBOBR HE 
BRR ORDEDR 
R@BOR BOR ABO 
RBOCRBDOR] 
R@BOBOR WOR 
CSCRBRBOKSOD 


SOOCDMR HBR He 


111 
lla 
118 
lal 
laa 
lag 
1p1 
1Ba 
168 


CRR HK ODBRDO| DRKHDBRKHR@WO! ORK 
ROMPDBHOORD! BR HKKHDWR ORD! OR 


DWREBR HBR HE 
RBDWR RQ RRH 
te 
RR OCCOBHEDOR 
RRBOR BOR WO 
SRkRRBOR BOR] 
WOR RDOR WOR 
RRR BRBRRBOCS 
REoBOOORARR 
RPODBOrR DBR He 
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The MRT and the RMRT can be subdivided into 
blocks by taking together all columns (and rows) with 
given numbers of ‘0’s in their labels. In Table I the 
blocks are indicated by heavy lines. We denote by H,, 
the set of k-tuples with m non-null elements, as well as 
the sets of columns and rows labeled by those k-tuples. 
In a block with rows in H,, and columns in H,,, each row 
has the same number u,,, of ‘0’s. In the Appendix we 
derive a formula for the numbers u,,,, which enables one 
to count these ‘0’s and learn something of the structure 
of the MRT for C?’ without writing out the whole table. 

In particular, the number of ‘0’s in a column of the 
MRT is M‘”’. Therefore, in an (M, n, k) alphabet the 
sum of the weights w, of all the letters is 


L 
>> w, = n(M* — M*”) = nM*\(M — 1) = BS, (2) 


and the nonzero weights form a partition of this number 
S into L — 1 parts (leaving out w, = 0). (For M = 2 
this is Slepian’s Proposition 6.°) 


IV. Maximum-Wetieut (M, n, k) ALPHABETS 


The distance between two code letters can be con- 
veniently defined as the number of places in which they 
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differ. The greater this distance, the less likely one lett 
is to be received when the other is sent. Now this distanc 
is the weight of a third letter of the alphabet, found b 
subtracting the two letters digit by digit. Therefore, tk 
(M, n, k) alphabet, constructed so that the weights 1 
of all letters other than J = (000 --- 0) are as large ¢ 
possible, can be expected to have nearly the smalle: 
attainable probability of error. We call such alphabet 
“maximum-weight alphabets.”’ 

If instead of quantizing each received signal into on 
of the M digits, the receiver bases its decisions about th 
transmitted letters on the amplitudes of the n-associate 
received signals, using the principle of maximum likel 
hood, the maximum-weight alphabet yields the large: 
possible probability of correct reception among all (M, n, | 
alphabets, at least in the limit of. large signal-to-nois 
ratio. When the received signals are first quantized int 
digits, n-tuples of which are then decoded into lette1 
of the alphabet, the maximum-weight alphabet does ne 
always yield the minimum probability of error, as Slepia 
has shown for the (2, 7, 3) code.* One expects, howeve! 
that the maximum-weight alphabet will be less vulnerabl 
to noise than most of the large number of possible (VM, n, kh 
alphabets. 

MacDonald,” and Bose and Kuebler’ have discusse 
the construction of binary group codes of maximul 
weight. Bose and Kuebler treat the problem in terms ¢ 
a finite projective geometry, and their approach coul 
no doubt be extended to M-ary alphabets when M ; 
the power of a prime. The first k elements of the K columr 
of the RMRT are the homogeneous coordinates of th 
points in such a finite geometry. 

We define a ‘‘maximal partition” of S = nM*"'(M — J 
into LZ — 1 parts as one in which all the parts are ¢ 
large as possible, without regard to whether a group coc 
with those weights exists—usually none does. If w, 
the smallest nonzero weight, it satisfies the imequalit 


w, = S/i— 7); 
and if w, is the next smallest (w, > w,), 
W, S (S — w)/(L — 2), 


and so forth. The maximal partition is easily calculate 
by taking the largest integers permitted by these ii 
equalities, 

If the letter length n is a multiple of K: 


fae 


n = Ks, K = (M* — 1)/(M - J), ( 
the maximal partition is one in which all weights are equé 
w; = sM*", P< 7s Mae ( 


For s = 1, alphabets having these weights can be forme 


kee MacDonald, “Design methods for maximum minimus 
distance error-correcting codes,’ IBM J. Res. Dev., vol. 4 ja 
43-57; January, 1960. i 
“ R. C. Bose and R. R. Kuebler, Jr., “A geometry of bina 
sequences associated with group alphabets in information theory, 
Ann. Math. Stat., vol. 31, pp. 113-139; March, 1960. | 
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y using as letters the K rows of the RMRT, along 
with the digitwise product of each row with the (M — 2) 
honunit field elements Stated otherwise, one picks one 
*olumn of the MRT from each of the K sets of columns 
liffering only by a multiplicative factor. For s > 1, one 
epeats each of these letters s times 

| When the length n is not a multiple of K: 


n= Ks+h, (5) 


Sra, SG el cet Ge 


he maximum-weight codes are formed by appending 
»ertain columns of the RMRT to the s-times repeated 
*olumns used for n = Ks These h extra columns are to 
xe chosen so that the weights of the rows of anh X K 
natrix formed from them are all as large as possible 
Phe h columns to be used do not depend on s, and we 
shall take s = 0. As before, the list is to be expanded to 
L — 1) letters by multiplying each row by the (M — 2) 
xonunit field elements to obtain the complete alphabet. 
here are many equivalent codes with the same sets of 
veights, for the best choices of columns are not unique. 
“Zach column, for instance, can be multiplied through by 
, nonunit field element to yield a new column that serves 
1s well. Still other transformations and permutations of 
-olumns are possible that yield equivalent codes. 
' We conjecture, but have not proved, that in the set 
»f h extra columns for a group code of maximum weight, 
10 column will appear twice There are exactly enough 
Riinns in the RMRT for this to be possible for all 
values of h. Plausibility is lent to the conjecture by the 
servation that one can usually pass from h = h’ to 
, = h’ + 1 by adjoining a previously unused column of 
a RMRT to the optimum set for h = h’, choosing 
she new one so that it has non-null elements in as many 
f the rows of minimum weight for h = h’ as possible. 
Towever, for certain values of k and h = h’ it has been 
ound necessary to drop a column and add two new ones 
n passing to h = h’ + 1. For still larger values of k, 
nore extensive revisions may be needed at certain values 
fh. 

In Table II we list, using Slepian’s notation,” a set of 
hoices of extra columns yielding maximum-weight codes, 
nd the resulting letter weights, for M = 2,k = 2, 3, 4. 
‘or instance, when k = 4, (124) stands for the binary 
umber (1101), and ‘“3”’ means that there are five 
otters of weight 3 in the alphabet. In Table III we give 
he extra columns and weights for M = 3, k = 2, 3, 
nd in Table IV we list the same for M = 4. For M = 3 
he digits are the integers modulo 3: 0, 1, 2; for M = 4 
re use 0, 1, a, and @ as in (1). 

We shall now briefly discuss the procedure for finding 
hese maximum-weight codes. For values of hin] <h < k, 
ne picks any fA different columns from the part of the 
-MRT labeled by H,. As h increases from k, one adds 
olumns from H, until these are exhausted, or until too 
reat an imbalance among the weights seems to occur. 
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As for M = 2,k = 4, h = 6, it may then be necessary 
to drop some columns and add new columns from other 
parts of the RMRT to find the best selection. 

For values of h near K, one proceeds by successively 
dropping columns from the complete RMRT. The best 
procedure here seems to be to choose two elements 
of C;’ not in H, and not differing by a multiplicative 
factor, and to form the subgroup they generate. One 
first drops those columns of the RMRT labeled with 
elements of that subgroup. One then takes an element 
not in the subgroup and forms the subgroup generated 
by it and the elements of the preceding subgroup. The 
next columns to be dropped from the RMRT are those 
labeled by the elements of this new subgroup, but not 
already removed. One continues thus until the largest 
subgroup has been formed and the associated columns 
removed from the RMRT. With possibly a few modifica- 
tions this procedure should yield maximum-weight codes 
fore Shak. 


APPENDIX 


DISTRIBUTION OF ‘0’S IN THE MopULAR 
REPRESENTATION TABLE 


Our problem is to find the number u,,,, of zeros in each 
row of a block of the modular representation table with 
row labels in H,, and column labels in H,, that is, the 
number of ways of obtaining 


ab, + dab. + --- a,b, = 0 


when n of the a,’s and m of the 6,’s differ from ‘0’. Since 
Hmn 18 the same for all rows in the block, we choose 


=--=b =0. 


Diet 


Then u,,, 18 the number of ways (a, + a2 + --- a,,) equals 
0 when (a;, a2, --- a,) isin H,,. 

We fix at r the number of non-null a,’s among the 
first m, with n — r remaining in the last k — m places. 


Later we sum over r. There are 
r k—™m n—T 
L ie _ :) us ) 


ways of choosing the a,’s to satisfy this condition; of 
these, a fraction 7,/(M — 1)’ yield a, + az + +--+ An = 0. 
Then (7) = 1) 


too = Seed) pO 


r=0 n 
We must now find 7,, which is just the number of ways 
S. Gy aie Oa a, 


can vanish when the r a,’s, | <7 <r, run through the 
(M — 1) non-null field elements independently. Let 7/ 
be the number of ways of getting 


8S =a tate--a=x¢rH0. 
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TABLE II 
Maximum-Wercur Group Coprs (VM = 2) 
Labels of Extra Columns Weights | Maximal 
h from MRT Wi Partition TABLE II 
e=a20 ili 3 Omi2 Oy 12 Maximum-Weicut Group Copzs (M = 3) 
2 1E2 12:2 122 
i= Sal il Ge te Oy? ie Labels of Extra Columns Weights 
2 1,2 Opi 22 The 24 h from MRT Wi 
3 JAAS 18233 12.25 
4 1, 2, 3, 123 26 4 25 32 k= 2 1 01 rls 
5 I, PS, A ale} 22 31 4 2735 2 01 + 10 142 
6 Le aalion 25 3! 48 34 43 3 01 +411 2832 
4 01 +12 38 
a 4 1 07 18 07 18 a 
: 2 1,2 03 18 24 142 (ao 1 001 08 118 
3 3 0 182832 16 29 2 001 + 010 02 112 28 
4 Ip OF ety a 1428344 SY 3 001 + 100 16 212 312 
& 1, 2, 3, 4, 1234 21045 25 310 4 001 + 111 212 38 46 
6 12: 2 4, 134, 234 23 38 436 31243 5 001 + 112 2? 34 46 5 
a il PA Gy ah 124, 134, 234 3° 477 34 All 6 001 + 120 34 418 64 | 
8} 1,2, 3, 4, 124, 134, 234 + 123 | 448 Aul5é i 001 + 121 410 52 62 
Chih il Pa ae 124, 134, 234 + 13 46 588 AS Fle 8 001 + O11 516 68 82 
i), Whee ees 124) 134, 234 + 14 4258648, 51065 9 001 + 102 624 92 
11 | 1, 2,3, 4, 124, 134, 234 + 23 586°728 5? 613 10 001 + 122 68 718 92 
12] 1, 2,3, 4, 124, 134, 234 + 24 61283 69 76 il 001 + 012 712 Biz Q2 
13 | 1, 2,3, 4, 124, 134, 234 + 1234] 6 78 83 GO 7s 12 001 + 101 818 98 
14 | 1, 2, 3, 4, 124, 134, 234 + 34 78 87 78 87 13 001 + 110 926 
TABLE IV 
Maximum-Wercut Group Coprs (M = 4) 
Labels of Extra 
h Columns from MRT Weights w; 
k=2 if Ol (0S ee 
2 01 +10 16 29 
3 01 +11 29 38 
4 Ol + la 32 43 
5 01 +18 41s 
k=3 1 001 Ou 148 
2 001 + 010 03 124 236 
3 001 + 100 19 227 327 
4 001 + 111 218 324 421 
5 001 + 18a 330 415 518 
6 001 + lap 445 618 
of 001 + 166 49 536 G6 712 
8 001 + Ola 618/639 788? 
9 001 + 101 627 727 99 
10 001 + 110 63 733 B18 93 106 
11 001 + laa 76 839 Qiz 116 
12 001 + 018 89 948 128 
13 001 + lla 921 108 123 138 
14 001 + lel 1088 1124128 148 
15 001 + 106 1145 1215 158 
16 001 + 160 1269 163 
17 O01 + O11 122% 1348 163 
18 001 + 10a 1374 1436 163 
19 001 + lad 1436 1524 163 
20 001 + 118 1548 1615 
21 001 + 161 1621 


Because of the symmetry among the non-null group 
elements, 7/ is independent of a. The following recursion 


relations connect 7, and 7/: 


Troi = (M ra Dare 


ll 


, 
Tr+1 


Tt, + (M — 2)7!. 


The former of these indicates that one can ee Syria 
in (M — 1) wares namely by taking a,,, = —x whe 
8, = x, « ~ 0; s, = x was formed in 7/ ways. Similarl 
one gets s,,, = « # 0 by adding a,,, = x tos, = 
or by adding the element x — y tos, = y ¥ 0. The form) 
can be done in one way, the latter in (M — 2) ways. 
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The solution of the above difference equations with The symmetry relation 
itial conditions 7, = 0, rf = 1is 


r= (M — DME — 1) — (- (i Jens — 0 = (P)peat a — a 


, = [Uf — 1)" — (-17 , 
- [( ) (—))")/M follows from the symmetry of the MRT. To find the 


hus we finally get numbers of ‘0’s in the blocks of the reduced MRT, 
k divide all the u,,,’s by (M — 1), 1 < (m,n) < k. 
== M (ag — 1)” For a binary MRT (M = 2), one has the simpler 
; generating function 
< m k SG r n-—rt 
+E(™\E Tt -per-ore} rn = aa tata tata an, 
It is often convenient to use the generating functions For k = 5, M = 2, for instance, the matrix || u,,, || is 
k ‘ : best MOM 10a Seek 
RO) = Do thm” = Mf [1 + (M — Ie} 
n=0 Bf Se OPA LyO 
sea ee kM 1)a(bee (Lz) ),, ipmuitesleia one es 
) particular, the total number of ‘0’s in a column or Piso Dens MGR O 
yw of the MRT is 
5 Le ADL RO sy Oe alin tt 
Ful) = 20 bmn = M", m = 0, 1: 0.10% 20 53002! 
SMe, m # 0. 
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CORRECTION 


The Editor wishes to call attention to the following 
corrections in G. R. Welti’s paper “Quaternary Codes 
for Pulsed Radar,’ which appeared on pages 400-408 
of the June, 1960, issue of these TRANSACTIONS. 


1) Page 402, column 2, next to last equation: Right- 
hand side should read 


° a | 

a Bl 

2) Page 405, column 1, four and one-half inches from 
bottom of page: The equation should begin with 


ae 
Lae 


3) Page 408, column 2, second equation: Left-hand side 
should read 


- Di + Di + Ai + Aj. 


556 


Correspondence 


Second-Order Properties of the Pre- 
Envelope and Envelope Processes* 


In a recent paper, Dugundji' introduced 
a generalized definition of envelope based 
on the notion of “analytic signal’ or “pre- 
envelope.’ Section IV of Dugundji’s paper 
contains the results: 1) The time auto- 
correlation function of the Hilbert trans- 
form of a waveform is equal to the time 
autocorrelation of the original waveform, 
2) The time cross-correlation function 
between a waveform and its Hilbert trans- 
form is the Hilbert transform of the time 
autocorrelation of the original waveform, 
3) The time autocorrelation of the pre-enve- 
lope of a waveform is twice the pre-envelope 
of the time autocorrelation function of the 
original waveform. For ergodic processes, 
the same results remain true, indeed, when 
time averages are replaced by ensemble 
averages. The purpose of this note is to 
show that the results mentioned above of 
Section IV in Dugundji’s paper remain 
true when time averages are replaced by 
ensemble averages, regardless of ergodicity, 
the only requirement being that the process 
be wide sense stationary. It will also be 
shown that a corollary of result 3), recently 
pointed out by Brown,? the envelope of an 
ergodic process has a variance which is 
twice the variance of the original process 
also remains true when the word ergodic is 
replaced by the words wide sense stationary. 

The key to the proofs given here is to 
consider the frequency domain representa- 
tion of the Hilbert transform. This approach 
simplifies the proofs since the frequency 
domain representation of the Hilbert trans- 
form is a simple all pass, nonrealizable 
“filter’’ or transfer function. 

Letting {u(t)} be a wide sense stationary 
process and S,(f) (with f° S.(f)df < ©) 
be the spectral density of the process, the 
transfer function Y(f) defines a linear 
operation on {u(t)} if and only if* 


[ \vor sa < 


if {#(t)} is the result of operating with 
Y(f) on {u(t)} then {2#(t)} is also wide 
sense stationary with the spectral density: 


Sf) = | YA) ? S.A) 


and the cross spectral density S,,a(f) is 


* Received by the PGIT, March 4, 1960; revised 
manuscript received, July 11, 1960. This work was 
supported by the Sci. Dept., Ministry of Defence, 
Israel. 

1 J. Dugundji, ‘‘Envelopes and pre-envelopes of 
real waveforms,’’ IRE Trans. on INFORMATION 
Tueory, vol. IT-4, pp. 53-57; March, 1958. 

2W. M. Brown, ‘Some results on noise through 
circuits,’ IRE Trans. on Circuir Turory, vol. 
CT-6, pp. 217-227; May, 1959. 

3 J. L. Brown, Jr., ‘‘A property of the generalized 
envelope,’’ IRE Trans. on Crrcurr THeEory, vol. 
CT-6, p. 325; September, 1959. 

4 J. L. Doob, ‘‘Stochastic Processes,’’ John Wiley 
and Sons, Inc., New York, N. Y., p. 534; 1953. 
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given by 
| Sud) = YNSA): 
The linear operation induced by 
fie 
0f=0,i=V-1 (1) 
OP eal Un 


Y(f) 


I 


—7-sgn(f) 


will now be defined as the Hilbert trans- 
form, the familiar time domain representa- 
tion of the Hilbert transform will be 
derived later from (1). 

1) The spectral density of {#(t)}, where 
{a(t)} is the Hilbert transform of {u(t)}, 
is, therefore, given by 


Sf) = | YO |’ S.A) 
Su, 


and therefore, the ensemble autocorrelation 
function of the {#(t)} process satisfies 


Ri(r) = Ril). 


2) The cross spectral density between 
u and @ is given by 


Suef) = YDS) 
= —isen (Sf) (2) 


the ensemble cross-correlation R,, (7) is, 
therefore, 


for { #0 


Ruel) = [ YOSAHe* rdf. 


Since | Y(f) | = 1 and S,(f) is integrable 
(—o, ~), R,,a(7) is defined for all 7, is 
bounded and continuous, and tends to zero 
as 7 tends to +o, 

As the Hilbert transform was defined in 
(1) in the frequency domain only, we have 
to show that in the time domain the relation 
between FR, and F,,,4 is given by 


R.A RD | 


a 


R,,.a(7) — 


where P denotes the principal value. In 
order to derive the last equation, let 


—t2rft dt 


“Hf e 
qv Al t 


A —i2nft 
+ [ c__& uy (A>s>0) 
6 


Y(f, 6, A) 


= —=!. (Si(2nf A) — Si(2rf 8)) 


Decembei 


where 
Woe Wen) 
site) = [ ay; 


then Y(f, 6, A) is uniformly bounded ir 
f, A, 6 and Y(f, 6, A) tends to Y(f) as 
A — o and 6 — 0. Therefore, 


R(x) = lim _ Ys 5, A)S,(fye?”’ rdf 


A> 0 = 


60 


Hence, by the convolution theorem, 
Ne itis Wil 

Rr) = lim : / RAD g 
Avo \T J—-A+r 


Ti ye 
6-0 
A+rT 
Tee) . 
TW Jb+r oak 


1 SLO) 
=5P[ sea 3) 


3) The pre-envelope process {u(t) + 
iu(t)} is obtained from {u(t)} by a linea 
operation with the transfer function [1 + 
iY(f)], where Y(f) is as defined in (1) 
hence, 


Sarimf) = |1+7¥/ |S.) 


= 21 + eY(AIS.A) 
(4) 


peek f>0 
= 425.(), f= 
pee 


and from (1), (2), (4), the ensemble auto- 
correlation function of the process {u(t) + 
in(t)} is given by 


Ry ia(7) = 2[R.,(7) + iR,(7)]. 


4) The envelope V(t) of u(t) is defined az 
V(t) (u(t) + 7#(¢)]!/2. Therefore the 
expectation of V2(¢) is given by 


E[V*(t)] = RO) + R.(0) = 22.0) .| 


Hence, the ensemble average of V*(¢) 
twice the ensemble average of w(t). | 

As for the time domain representation 
the operation induced by Y(f) in (1) with 
respect to the process {u(¢)}, using th 
same approximation as in the derivation 
of (3), it follows that a(t) is the limit ii 
the mean of 


ae uz). 
Tv ais A t— 7 a7 pie fs 
as A > ~, 6 > 0. 

The representation of the Hilbert trans: 
form as a (all-pass, nonrealizable) “filter 
or “transfer function’’ was used in this not 
to show that some known properties of t 
pre-envelope and envelope of ergodic proo 
esses are actually second-order propertie 


er. 
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7.€., they hold for any wide sense stationary 
process. This representation can also be 
used to simplify the derivation of some 
other results on Hilbert transforms and 
pre-envelopes. The frequency domain analy- 
‘sis has an additional advantage over time 
domain analysis in this case, since it avoids 
Manipulations with improper integrals. 


Mosse ZaKAt 

Sci. Dept. 

Ministry of Defense 
Israel. 


Upper Bounds for Error Detecting 
and Correcting Codes* 


The excellent paper submitted by N. 
Wax,! prompts me to contribute some 
additional thoughts for possible extension 
of the work on this subject. 

From an examination of Table I, ‘“Col- 
lected Results,” in the original paper, the 
Hamming bound is apparently superior 
for low redundancy, or, equivalently, for 
small e/n ratios, whereas the improved 
“soft sphere’? model is better for a range 
of moderate error correction. 

This letter describes an approach to an 

error-correcting bound that can be made 
smaller than either of the aforementioned, 
under the condition that the number of 
correctable errors is large for a given n. 
Although a rigorous proof is not included 
in this letter, a logical development. is 
offered. , 
The described bound, which may be 
referred to as the maximal codistant 
bound, arises from an argument that is 
in contrast to the usual “‘sphere’’ geo- 
metrical model; 7.e., Hamming? packs » 
disjoint spheres of error-correcting radius 
e to produce a minimum 7» space—of 


v 9S. (") KU. 
i=0 \U 


— On the other hand, Wax’s bounds use 
clever geometrical techniques, which, as 
an analog, use rigid and elastic spheres in 
an attempt to fill the n space, and thereby 
arrive at a better estimate of the informa- 
tion density. 

The maximal codistant model assumes 
a given n space and a number of message 
points, v, to be disposed in such a manner 
as to yield a maximal mutual separation. 
By maximizing the minimum of such 
distances, a corresponding maximum error- 
correcting capability will be indicated. The 
reasoning proceeds on the basis that, in 
2 given n space, the sum of all combina- 
tions of all distances between the v message 
points (for a binary field) can not exceed 
n/4. This magnitude is obtained when 
she ith digit of all v words is examined. 


: 


* Received by the PGIT, May 20, 1960. 

1N. Wax, ‘“‘On upper bounds for error-detecting 
nd error-correcting codes of finite length,’ IRE 
"RANS. ON INFORMATION THEORY, vol. IT-5, p. 
68; December, 1959. 

2R. W. Hamming, ‘‘Error-detecting and error- 
orrecting codes,’’ Bell Sys. Tech. J., p. 147; April, 
950. 
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TABLE I 
Co..ectep REsutts 
R? 0.75 1.25 1.75 2.25 2.75 3.25 
n e 1 3 2 4 5 6 
vi V1 74.2 12.4 4.1 
UH 16 4.4 2 
v2 18 .2* Sol 2.1 
Ve _— 3.33 2 
Ng 16 ?4 Pa 
8 V1 199.3 25.6 vies | 
OH 28.4 6.9 2.8 
v2 20 .3* 5.3 3 
Ve == 5 2.34 
NB 20** 4 2 
9 V1 56.7 56 .2 12.9 4.8 
OH 51.2 AL ea | 3.9 2 
v2 39 .7* 9.2 ax Diels 
Ve —— 10 2.8 2 
Ng 38** 6 2 2 
10 v2 1.69 X 103 128.7 25.0 7.6 
OH 93.1 18.3 5.8 PIRI 
v2 82.2 16.6 6.1 3.4 
Ve — — Sg) 2.29 
Ng 68** 12 2 2 x 
11 V1 5.29 X 103 312.5 50.6 13.5 5.5 
vA 170.7 30.6 8.8 3.6 2 
v2 154 .8* 26 .5* 8 .6* 4.3 2.8 
Ve _ _— 4.67 2.58 2 
Ng 128 24 4 D 2 
12 V1 1.72 X 104 781.2 106.9 25.5 8.8 
0H 315.1 51.8 US 5.2 2.6 
v2 346 .8* 46 .7* 12 .7* ext 3.3 
Ve — — 7 3 2.2 
Neg 256 24 4 2 4 
13 V1 5.84 X 104 2.05 X 103 235.8 48.9 15.2 6.3 
UH 585.1 89.0 PANE CE 75 3.4 2 
v2 806* 85 .1* 19 .3* v2 4.0* Pe 
Ve — — 14 3.6 2.44 2 
Neg 512 32 8 2 Pe 2 
14 V1 2.05 X 105 5.56 X 103 537 .6 97.6 26.8 10.0 
oH 1.09 X 103 154.6 34.9 jah ea 4.7 2.5 
V2 1913* 160* 30 .1* 9 .8* 4 .9* SEG) 
Ve = — _ 4.5 2.76 2.16 
Neg 1024 48 16 4 2 2 
15 V1 7.44 X 105 1.55 X 104 537 .6 97.6 26 10.0 
UH 2.05 X 108 270.8 56.9 16.9 6.6 3.0 
v2 4656* 309 .8* 48 .9* 13.8* 6.2* 3.9 
Ve — — — 6 3.14 2.36 
Ne 2048 = 32 4 2 2 
16 V1 2.78 X 108 4.47 X 104 3.11 X 103 431.0 94.0 28.5 
oH 3.86 X 103 478.4 94.0 26.0 9.5 4.4 
v2 1.16 X 104* 617 .3* 82 .0* 24.2 8.8 4.7 
Ve — a —_: 9 3.67 2.6 
Np 2048 = 32 as 2 2 
17 V1 1.07 X 107 1,583. >< 105 7.81 X 103 951.5 185.2 50.7 
OH 7.32 X 103 851.1 15722 40.8 13.9 6.0 
v2 2.96 X 104* 1264* 140 .0* 37.5 re 5.9 
a eee a= = 18 4.4 2.9 
Np 4096 == 64 at 4 2 


vy == the Hamming bound. 
ve = the Maximal Codistant bound. 


v1 ~= the upper bound obtained from the “‘hard sphere’’ model. 
= the improved upper bound, using the ‘‘soft sphere’’ model. 


Ng = the best value actually found. These entries have been taken largely from Laemmel, except for 


entries marked with a double asterisk. 


To assure a maximum sum of distances 
(in the ith column), there must be an 
equal number of ones and zeros. (Hence, 
a necessary but insufficient condition for 
meeting the bound is that v be even.) 
This produces the sum (v/2)?. 

This is then increased by the factor n 
to account for all dimensions. The next 
step is to let such total distance be dis- 
tributed, so that all mutual distances are 
equal. This assures that the minimum 
distance will be maximized. Since there 


v Sette, 
are | 5 combinations of “paths’? among 


v points, the minimum distance between 
message points cannot exceed nv/[2(v — 1)]. 
Hence, e- < nv/[4(v — 1)] — 4 would be 
the upper limit for error correction. It will 
be observed that the error-detection upper 
bound, ea < nv/[4(v — 1)], can be realized 


exactly in many existing codes. v need not 
necessarily be a power of 2 (as in a syste- 
matic code) for meeting the bound. (An 
example is n = 11, v = 12, and e = 3. 
This is achieved by using all eleven cyclic 
shifts of the code word, 11101101000, and 
the zero vector.) Note also that for reason- 
ably large v, the well-known result of 
lim,s. e@ = n/4 immediately becomes 
apparent. 

As stated previously, the maximal co- 
distant bound applies only to codes in 
which e is relatively large. This is plausible 
from the standpoint that, if the space is 
sufficiently large, a few message points 
can always be disposed to meet, almost 
or exactly, the codistance symmetry re- 
quirement. 

The range over which the maximal 
bound appears to be equal or superior to 
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the Hamming bound is 2e + 1 <n < 
4e + 1,e #1 and the trivial case, 2e + 1 < 
nm <4e,¢ = 1. 

_ The table in Prof. Wax’s paper is 
herein duplicated and supplemented to 
include the maximal codistant bound. 
In this extended form, the inequality 
becomes v. < (4e + 2)/(4e + 2 — n). 
Again, the limits on » = f(e) have been 
observed, which limits the entries only 
to part of the table. 

In order to highlight the ranges over 
which each of the bounds excel, the best 
limit on v; for given n and e is shown in 
Table II. 


TABLE II 
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H—Hamming’s Bound. 
S—Wax’s Soft Sphere Bound. 
C—Maximal Codistant Bound. 
Peston that both bounds yield the same 
result. 
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A Note on Error Statistics in Fading 
Radioteletype Circuits* 


A great deal of study has been devoted 
in recent years to the problem of detecting 
binary data which are transmitted through 
noisy, fading channels. Nearly all published 
theoretical investigations, however, have 
been concerned only with minimal first- 
order error probabilities. There is at present 
considerable interest in applying error- 
correcting codes to fading radio circuits, and 
this raises the question of higher-order 
error probabilities. A sample computation 
of one such statistic is given below, and its 
relevance is discussed briefly. 

Reiger and others have shown! that 
when additive white Gaussian noise is the 
only form of disturbance, the error prob- 
ability in noncoherent, matched-filter de- 
tection of frequency-shift-keyed teletype is 


where r; is the signal-to-noise power ratio. 


* Received by the PGIT, February 15, 1960. 
The work discussed in this letter was done in 1957— 
1958 at the Long Range Radio Branch of the 
USASRDL, Fort Monmouth, N. J. 

18. Reiger, ‘Error probabilities of binary data 
transmission systems in the presence of random 
noise,’ 1953 IRE Narronan Convention ReEcorp, 
pt. 8, pp. 72-79. 
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If the signal is subject to nonselective 
fading, r; must be considered as a random 
variable and a mean value for the error 
probability can be found by averaging over 
all values of 71. For fading which is Ray- 
leigh-distributed, the probability density 
function (pdf) of r1 is 


1 Ti 
pr) = 2 exp | |, US (2) 


where p is the mean value of the fading 
signal/noise ratio. Thus the mean error 
probability under Rayleigh-fading con- 
ditions is? 


P(«) 
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Clearly, this result applies only to sets 
of decisions whose members are spaced 
sufficiently far apart in time, frequency, 
or space so that they are essentially inde- 
pendent; it certainly does not apply to 
contiguous data pulses which are trans- 
mitted at a rate considerably faster than 
the ‘‘speed” of the fading. 

To evaluate the average first-order con- 
ditional error probability in slow-Rayleigh- 
fading situations, one must find the average 
probability of two adjacent errors. This 
probability, given the signal-to-noise ratios 
at the two detection instants, is 


(3) 


z 
P(E, € | 1,12) = ¥ exp | 


*} exp | I. (4) 


An average value for this error-doublet 
probability can be found by weighting (4) 
with the bivariate pdf of r1 and re, which iss 
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(The parameter ¢ which appears in (5) 
is the correlation coefficient over the pulse 
length of the quadrature components of 
the fading signal. In essence, ¢ is a measure 
of the “speed”’ of the fading. When ¢ = 0, 
the fading rate exceeds the data rate, 
while when ¢ = 1, the fading is very slow 
indeed.) The average probability of two 


(5) 


2M. Masonson, ‘‘Binary transmissions through 
noise and fading,” 1957 IRE Narronan ConvEen- 
TION REcorD, pt. 2, pp. 69-83. 

3S, O. Rice, “Mathematical analysis of random 
noise,”’ in ‘Selected Papers on Noise and Stochastic 
Processes,’’ N, Wax, ed., Dover Publications, Inc., 
New York, N. Y., pp. 133-294; 1954. 
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adjacent errors is 


P(e, @) = [2 fs p(r1, 72) 
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The average first-order conditional erro 
probability is 


(6 


(ise (7 
p(l—¢g)+4p+4 * 


When ¢ = 0 (fast-fading), the conditiona 
error probability approaches the first-orde: 
error probability given by (3). This is ¢ 
condition which violates the assumption: 
underlying (1), and which is almost neve! 
met in practice. When ¢ > 0, the con 
ditional probability always exceeds th 
first-order probability. It is interesting tc 
put some real-life numbers into (3) anc 
(7). For p = 30 db andg = 0.995, whicl 
are values characteristic of high-frequency 
long-range radio practice: 


Py 10e 
Ple | €:) ea 7, x< 10° 


These values imply that error-burst-cor- 
recting codes may be useful for fading 
radio circuits. 

The preceding results are the simplest 
of many which can be derived. Recurrence 
formulas can sometimes be developed 
(depending on the correlation character- 
istics of the fading) to reduce the multiple 
integrals which arise in evaluating multiple- 
error clusters, and combinatorial analysis 
can be used to determine what types ol 
error-clusters warrant consideration within 
given redundancy constraints. There are 
however, serious limitations on the results 
given here, and on direct extensions thereto 
For example, (6) and-(7) are valid only 
for white noise interference and Rayleigt 
fading, and apply only to nondiversity 
reception. Furthermore, they are limitec 
to FSK systems with narrow frequenc 
shifts, for an underlying assumption i 
that fades in the “mark” and ‘space’ 
sub-channels are completely correlated. I 
this assumption is not made, tractabl 
general results can’t be derived readil 
because the conditional probabilities wi 
depend upon what character is being sen 
Unfortunately, the two classes of service 
wherein FSK is most widely used—h. 
layer propagation and ionoscatter propaga 
tion—do not conform to these limitations 
In the former, stationary white noise i 
an inadequate model for additive dis 
turbances, and in the latter, wide frequeney 
shifts are normally used. Thus, ealcula 
tions of the sort discussed here may proy: 
to be mainly of academic interest. 

The writer wishes to thank officials a 
the USASRDL and the Department o 
Defense for permission to publish thi 
letter. 
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This section of the issue is devoted to abstracts of material which may be of interest to PGIT members. Sources 
are Government, Industrial and University reports, and books and journals published outside the United States. 
Readers tamiliar with material of this nature which is suitable for abstracting are requested to communicate the 
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Chain Codes and their Electronic Appli- 
cations—F. G. Heath and M. W. Gribble 
(in English). (EE Monograph No. 392M; 
July, 1960.) 


A “chain code’’ is one using fixed-length 
groups of n binary digits, where each group 
uses n — 1 consecutive digits of the previous 
group shifted along one place, and one 
new digit constructed by some logical 
rule from the digits of the previous group. 
The groups can be constructed consecutively 
by a combination of a shift register with 
logic circuits for constructing the one new 
digit from the old ones. This method of 
construction bears some resemblance to 
the construction of check digits in Hamming 
codes, and chain codes can be used for 
error detection and for error correction on 
the basis of exhaustive comparison and 
minimum number of erroneous digits. 

Chain-code generators also serve as 
counters, or for frequency division of 
variable-frequency input pulses. 


Distortion Distribution and Error Proba- 
bility in Binary Transmission Systems— 
Y. Hoshiko, T. Minami, and T. Omori (in 
Japanese). (J. Inst. Elec. Comm. Engrs. 
Japan, vol. 438, pp. 146-153; February, 
1960.) 


The most important factors which cause 
error in binary transmission systems are 


I. Cederbaum 
Ministry of Defence 
Box 7063, Hakirya 
Tel Aviv, Israel 


noise and transmission distortion. In this 
paper, we discuss the residual response 
(intersymbol interference) at sampling 
points and analyze the general form of the 
distribution function of the residual response 
of a random binary signal pattern. This 
distribution function is ultimately given 
in terms of the characteristic transmission 
distortion function. The over-all effect of 
both transmission distortion and noise can 
then be obtained by convolution of distri- 
bution functions. A numerical example is 
given using the distortion of a constant-K 
filter circuit, and the error rate is computed 
at various signal-to-noise ratios. 


On (sin x)/x Sampling in the Case of 
Non-Band-Limited Functions—K. R. John- 
son (in English). (Lincoln Lab., Mass. Inst. 
Tech., Lexington, Mass., Tech. Rept. 195; 
February 16, 1959.) 


A theoretical investigation is presented 
of the use of (sin x)/x sampling to represent 
a function f(t) having a Fourier transform 
g(w) such that for large | w |, | g | and 
| dg/dw| tend to zero as | w |-%: and|w |-@:, 
respectively, when am > 2, a > 2. An 
upper bound is given for the error 
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as a function of w, and parameters describing 
the behavior of the transform of f. It is 
shown that under the above indicated 
conditions « approaches zero uniformly in 
t as ws approaches infinity. Also, an upper 
bound is obtained for ¢ for the case of sum- 
mation over only a finite number of sample. 
points. The uniformity of the convergence 
of the representation as the number of 
sample points increases is investigated. In 
addition, there is a rigorous proof of the 
(sin x)/x sampling theorem for band- 
limiting continuous function in Jy. : 


Representations of Vector-Valued Ran- 
dom Processes—H. J. Kelly and W. L 
Root (in English). (Lincoln Lab., Mass, 
Inst. Tech., Lexington, Mass., Group Rept. 
55-21; March 7, 1960.) 


In this note we extend two representatior 
theorems, well-known for ordinary random 
processes, to the case of vector-valuec 
random processes which are continuous in 
mean, square. The process to be representec 
may have finitely or, if certain convergence 
conditions are satisfied, infinitely many 
components. The first of these extensiona 
is an orthogonal series representation which 
holds on a finite interval. It is closely 
analogous to the Karhunen-Loéve repre’ 
sentation for complex-valued random proc) 
esses, and we prove its validity by ai) 
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rgument which parailels a standard proof 
)f the Karhunen-Loéve theorem. A 
yeneralization of Mercer’s theorem for 
jontinuous, definite kernels is required, 
}ut it also can be proved by the same 
J rguments as in the classical case. © 

| The second generalization is a vector 
jpectral representation theorem, holding 
nm the entire real line, under the weakest 
jatural assumption, namely, harmoniz- 
bility. This is not simply a reapplication 
if the usual theorem to a different space, 
put, like the orthogonal series representa- 
jon, it requires a slight extension of the 
previous results. 

+ The representations derived here promise 
tc be useful in signal-detection-in-noise 
Problems where the detecting apparatus 
128 more than one sensor, as for example 
the analysis of a “‘space-diversity’ radio 
‘ew radar receiver with multiple antennas. 
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Error Probabilities for the Ideal Detec- 
tion of Signals Perturbed by Scatter and 
Noise—R. Price (in English). (Lincoln Lab., 
sMass. Inst. Tech., Lexington, Mass., 
(sroup Rept. 34-40; October 3, 1955.) 


» Previous studies have shown how the 
junctional form of the ideal, probability- 
homputing receiver may be found for a 
lbommunication system in which one of a 
umber of possible bandpass waveforms is 
fransmitted over a channel containing one 
br more scatter-paths and additive white 
jJaussian noise. In this paper, a measure of 
the over-all performance of such a system, 
pmploying the ideal receiver, is found by 
Assuming that a decision about which 
waveform was transmitted is made on the 
asis of the a posteriori probabilities com- 
puted by the receiver. The probability of 
error P,,(e) in this decision is taken as the 
system performance, and, since it is the 
minimum probability of error obtainable 
oy any receiver for the given transmitter 
and channel, this performance measure 
may be considered to be a property of the 
sransmitter-channel combination alone. 
Only for the rather restricted case of 
on-off transmission through a single path 
aave solutions been obtained for P,,(e), 
and even then only in terms of the eigen- 
values of a homogeneous integral equation 
mvyolving the envelope of the transmitted 
waveform and a function that yields the 
somplete statistical description of the 
scatter-path. In cases where the path is of 
a very simple Markoff type and the 
envelope is either constant for a finite 
interval or exponentially decaying over an 
infinite time interval, the integral equation 
has been solved, and specific numerical 
results for P(e) have been obtained after 
rather laborious computation. In the 
limiting cases of very slow and very fast 
scatter-path fluctuations, it has been 
possible to obtain good approximations to 
P(e) for a wider variety of envelopes and 
scatter-path statistics. 

In the special cases previously mentioned, 
sufficient results have been obtained to 
demonstrate two interesting characteristics 
of scatter-path transmission which may 
hold true in general. First, for sufficiently 
small additive noise, fast scatter-path 
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fluctuations yield better system perform- 
ance than slower ones. Second, in the case 
of a constant transmitted envelope, in- 
creasing the duration of the transmitted 
waveform in the same ratio that the 
additive noise is decreased leaves P,,(e) 
unchanged for slower scatter fluctuations, 
while the duration varies as the square of 
the noise for fixed performance when the 
scatter is fluctuating rapidly. 

It has been possible to draw some com- 
parisons between the performance of the 
system employing the ideal receiver and 
the best performance obtained from systems 
having identical transmitter-channel com- 
binations but different receivers. In one 
case, the performance has been found for 
a rather fictitious receiver that is somehow 
supplied directly with complete information 
about the scatter-path fluctuations as well 
as with the received waveform. Naturally, 
this performance is better than that 
obtained with the more realizable ideal 
receiver, and this improvement is shown 
to increase with faster fluctuation of the 
scatter-path. Two other receivers that have 
been studied employ a correlation-envelope 
detector and an energy-measurement de- 
tector, respectively. The former is shown to 
approach ideal performance for slow 
scatter fluctuations, while the latter ap- 
proaches the ideal for fast scatter fluctua- 
tions. 

An incidental result of this analysis is 
the determination of the probability distri- 
butions of finite-time energy measurements 
made on narrow-band Gaussian noises. 
These distributions are given in an appen- 
dix, together with details of thier computa- 
tion. 

A large portion of the work carried out 
in this paper applies directly to the interest- 
ing problem of determining with greatest 
efficiency whether or not a noise-like, 
Gaussian signal, or narrow bandwidth is 
present in a background of additive white 
Gaussian noise. Such problems are en- 
countered in radio-astronomy and molec- 
ular spectroscopy. 


Some Aspects of the Relative Efficiencies 
of Indian Languages—B. 8. Ramakrishna, 
et al. (in English). (Dept. of Elec. Comm. 
Engrs. Ind. Inst. Sci., Bangalore 12, South 
India, Tech. Rept. 1; 1959-1960.) 


This monograph describes some recent 
statistical studies in the Indian languages 
Hindi, Marathi, Tamil, Malayalam, Telugu, 
and Kannada. The keynote of this study 
is that where different languages can be 
employed to serve the same end purpose, 
their relative performances as alternate 
means of communication can be compared 
on the basis of the principles of information 
theory. Although this report is restricted 
mainly to the Indian languages, the con- 
cepts and techniques developed may find 
wider application. 

The study is based on two notions: The 
first of these is that for messages in different 
languages having the same meaning, their 
total entropies form inverse measures of 
the relative efficiencies with which the 
languages encode semantic content into 
linguistic symbols. Using samples of texts 
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in one language and their translations in 
another as semantically-equivalent ma- 
terials for comparison, the authors develop 
an information theoretical model of trans- 
lation as a process which leaves the semantic 
content (but not the entropy) invariant, 
and carries with it at the same time a 
certain noise in the semantic sense. On 
the basis of this model, the number of 
selective bits of information in several 
Indian languages semantically equivalent 
to one bit in English have been obtained. 
The implication of these results in respect 
of the choice of a suitable language for 
telegraphic communication is also discussed. 

The second notion introduced here con- 
siders different scripts as alternate means of 
transcribing phonetic content into written 
symbols. It is argued (with examples 
chosen from the history of writing) that 
the speed with which a script can be written 
is one of the significant criteria of its merit. 
In the terminology of information theory 
this amounts to saying that between two 
scripts the one which takes a lesser time 
to write a given phonetic content has a 
larger channel capacity. The question 
whether the adaption of Roman script to 
Indian languages in place of their current 
scripts based on syllabary results in speedier 
or slower writing of these languages is 
then considered. 

A subjective test was designed as follows: 
A number of individuals who could write 
both the Roman and an Indian script 
were asked to copy a given set of nonsense 
syllables in both the scripts, and the times 
taken by them were measured. An analysis 
of variance was then performed to arrive 
at an estimate of the intrinsic relative time 
requirements of the Indian and Roman 
scripts independently of the practice of 
the individuals in the two scripts. The 
indications of the preliminary results are 
that Tamil, Kannada, and Hindi may save 
about 5 per cent of the time by using Roman 
script, while Telugu and Malayalam may 
take about 5 per cent more time when 
written in Roman script. 

The monograph also gives information 
concerning the relative frequencies of the 
different speech sounds, different types of 
syllables, and the methods of computation 
employed. 


Codes with Zero Correlation—D. N. 
Tompkins (in English). (Engrg. Division, 
Hughes Aircraft Co., Culver City, Calif., 
Tech. Memo. 651; June, 1960.) 


Codes of ternary digits are presented in 
which each code possesses zero cyclic 
autocorrelation except when perfectly cor- 
related. The lengths of these codes are 
up to 19 digits. 

The codes are found by a technique 
which generates lists of binary sequences. 
Each list is prime in the sense that no 
sequence in the list is a cyclic permutation, 
an order inverse or an amplitude inverse 
of any other sequence in the list. It is 
shown that the size of such a list approaches 
an exponential behavior as the lengths of 
the sequences are increased. 

Application of the codes to specific areas 
such as pulse compression, Doppler meas- 
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urements, and noise synthesis is discussed. 
Since cyclic rather than linear correlation 
is requisite, its implementation is also 
presented. 


On the Detection and Estimation Prob- 
lem for Multiple Nonstationary Random 
Processes—J. K. Wolf (in English). (Dept. 
of Elec. Engrg., Princeton University, 
Princeton, N. J., Ph.D. dissertation; 
October, 1959.) 


Two major problems in the study of 
statistical communication theory are the 
detection and estimation of signals cor- 
rupted by additive noise. In this investi- 
gation, certain aspects of both problems 
are examined with particular emphasis on 
the processing of multiple, nonstationary 
time series, z.e., correlated random proc- 
esses, the statistics of which vary with 
time. 

As a preliminary to this study, some 
properties of random processes are ex- 
amined, including an introduction to the 
concepts of stationarity, uniformity, ergodi- 
city, and time-varying power spectra for 
nonstationary processes. Some theorems 
related to the optimum filtering of non- 
stationary signals are also examined. 

A general formulation of the multi- 
dimensional linear filtering problem is 
presented next and is used in’ finding the 
minimum mean-squared error,‘ linear time- 
varying filters for multidimensional inputs. 
Both polynomial signals and signals which 
are sample functions of random time series 
are considered. The solution to certain 
types of matrix integral equations which 
occur in this work are discussed. 

The subject of random systems is 
examined from two different viewpoints. 
In the first case, a study is made of the 
filtering of signals which have been trans- 
mitted through a linear filter with certain 
random transmission characteristics. Both 
time-varying linear filters with random 
parameters and linear filters with impulsive 
responses which are sample functions of 
random processes are discussed. In the 
second case, the design of optimum systems 
is studied when the components used to 
construct these systems have values which 
vary from their nominal values in some ran- 
dom fashion. It is shown that the optimum 
nominal values of such non-ideal compo- 
nents will differ from the values of perfect 
components except under certain special 
conditions. 

Next, the problem of detecting signals in 
noise is considered for the multiple input 
model, where each of the inputs can con- 
tain one out of many possible signals. The 
detection procedure for this model becomes, 
in general, the testing of multiple hy- 
potheses. Two detection criteria are ex- 
amined for choosing between multiple 
hypotheses and it is found that for both 
criteria, the decision is based upon the 
calculation of the likelihood functions for 
the various signals. Systems for calculating 
these likelihood ratios are then examined 
for deterministic signals, with and without 
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random parameters, and for signals which 
are sample functions of random processes. 
A multidimensional matched filter is 
introduced, and its relationship to the 
detection problem is examined. The choice 
of signals which minimizes the probability 
of a wrong decision is found for the case 
when there can be only two possible signal 
combinations at the input. 

Finally, a useful expansion is presented 
for stochastic processes which are functions 
of two variables. The utility of this expan- 
sion in specifying optical filters for the 
detection of signals in noise is examined. 
The defining relationship for an optical 
matched filter is derived and related to 
this optical detection problem. 


Vector Stochastic Processes in Problems 
of Communication Theory—E. Wong (in 
English). (Dept. of Elec. Engrg., Prince- 
ton University, Princeton, N. J., Ph.D 
dissertation; May, 1959.) 


Some aspects of the application of 
multidimensional stochastic processes to 
communication theory are studied. A 
standard notation and a survey of mathe- 
matical techniques are introduced at the 
outset, in order that continuity and con- 
sistency be maintained in the remainder 
of the text. 

The prediction and filtering of multiple 
stationary time series are studied. Particular 
emphasis is given to the methods of solution. 
The matrix factorization procedure due to 
Wiener and Masani is extended to include 
the continuous case. An intuitive derivation 
analogous to that of Bode and Shannon is 
given. A class of two-dimensional problems 
is defined where the factorization of the 
spectral density may be avoided. 

The maximum likelihood estimation of 
continuously modulated vector Gaussian 
processes is derived. The derivation uses a 
multidimensional orthogonal expansion due 
to others. Two practical schemes of modula- 
tion, quadrature modulation and single 
sideband, are discussed in this framework. 

The joint probability distribution of 
quadratic functionals involving vector 
Gaussian processes is studied. The char- 
acteristic function for the joint distribution 
is found in terms of the eigenvalues of a 
homogeneous matrix integral equation. In 
special cases, this problem has been studied 
by others. Reduction to their results is 
demonstrated. 

A relationship between the Fokker- 
Planck equation and an expansion of 
second-order probability density functions 
is developed. This relationship is shown to 
define a class of stationary Markoff proc- 
esses which have useful properties. 

Two problems are considered where 
disturbances in addition to additive noise 
play an important role in the reliability 
of communication. First, the problem 
where the signal has been passed through 
a network with random parameters is 
analyzed. Second, the effect of uncer- 
tainties in connections is considered. In 
both cases the minimum mean-squared 
error filters are derived. 


Decembe 


The following papers were publishe 
singly by the Professional Group on In 
formation Theory (I) and the Professionz 
Group on Automata and Automatic Contre 
(A) of the Institute of Electrical Communi 
cation Engineers of Japan, 2-8, Fujimicho 
Chiyodaku, Tokyo, Japan. All are i 
Japanese, but English abstracts are gives 
below when available. The affiliation o 
the author is given so that intereste 
readers may contact the author directl] 
for further information. 


On a Distance-Preserving Code Systen 
(I; May 20, 1960)—M. Fujii. (Electro 
technical Lab., 1, 2-chome, Nagata-cho 
Chiyuda-ku, Tokyo.) 


This code system is constructed of ¢ 
finite number (n) of codes, expressed by 
binary symbols, and is characterized a: 
follows: the codes are ordered in the 
manner of n successive integers; and for ¢ 
given positive integer not greater than n, 
1) if the value of the ordinary distance 
between two integers (mentioned above’ 
is smaller than the given integer, ther 
this distance is equal to Hamming distance 
between codes corresponding to both 
integers, and 2) if the former is not smalle1 
than the given integer, then it can only be 
seen that the latter is also not smalle 
than the given integer. 

Such a code system as mentioned above 
is devised for a treatment of the information 
in pattern recognition. The conception of 
the restricted-distance preserving code 
system is explained and a method of its 
construction is clarified. 


Zero-Crossing Information of an IF 
Speech Wave and its Band-Compression 
Systems (I; June 24, 1960)—K. Hirmatzu. 
(Tokyo Electrotechnical College, 2-2 Kanda 
Nishiki-cho, Chiyoda-ku, Tokyo.) 


The syllable articulation of “‘carrier- 
leaked’’ SSB clipping is as high as that oz 
SSB, approximately 90 per cent. Hence 
carrier-leaked SSB clipping is available 
as a new information source for communi- 
cation systems. | 

This paper indicates that the zero! 
crossing information in carrier-leaked SSF 
is a kind of special sampling of the speecl} 
wave, and that the carrier-leak level giver 
the properties of this information. Namely) 
the information in low-level carrier-leaked 
SSB clipping is similar to that of narro 
band FM, but that of high-level, to wid 
band FM. One is applicable to a speech 
level dynamic control system, the other te 
various speech-parameter extraction 
band-compression systems. 


Optimum Recognition Systems for Mar 
off Processes (A; July 7, 1960)—K. Hori 
chi. (Waseda University, School of Scien 
and Engrg., 1-647, Totsuka-cho, Shinjuk 
ku, Tokyo.) 


Every automaton has a system or org 
of perception in a broad sense and recognize 
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e perceived objects by its allowable 
nection. In practice, the individual percept 
idergoes some distortions or deteriora- 
ms from the conceptional pattern stored 
advance in the brain or in some equiva- 
1t system for recognition. Furthermore, 
en if all the given percepts are exactly 
e€ same as the conceptional ones, the 
evitable imperfection of the organ gives 
e to some deteriorations of the perceived 
tterns. In this connection, there exists 
ambiguity of recognition, and we can 
‘roduce probability theory into this field. 
For most cases, the recognized objects 
> given as the time series of a Markoff 
ocess. Therefore, any optimum system 
uich recognizes such objects with mini- 
im error can be constructed by means 
_ suitable considerations of the prob- 
ility properties of these time series. 
in this report a general treatment of 
sognition, a system in which the expected 
k of misrecognition is minimum for a 
» of losses preassigned to each allowable 
se, and a system with minimum prob- 
ility of misrecognition are studied in 
tail from the standpoint of statistical 
cision theory for the Markoff process. 
The optimum systems consist of select- 
switch circuits with some memory, 
insitional probability calculators, ampli- 
fs, comparison circuits, etc. Explicit 
amples are illustrated for electric circuits. 


Srror Rates in Binary Pulse Transmis- 
m (I; April 22, 1960)—Y. Hoshiko and 
Sugiyama. (Electrical Communication 
b., 1551 Kichijoji, Musashino-shi, 
kyo.) 


he error in a binary transmission system 
caused chiefly by the transmission 
tortion and the noise. As for the former, 
probability-theoretical problem of the 
ersymbol interference is a very compli- 
sed one, and hitherto no analysis has 
on made. Here we treat the analysis 
this problem, then consider the effect 
tthe noise, and finally calculate the over- 
error probability in the system. 


Abstracts 


The Normalization of Patterns (I; July 
15, 1960)—S. Inomata. (The Electro- 
technical Lab., 1, 2-chome, Nagata-cho, 
Chiyuda-ku, Tokyo.) 


A computational algorithm which nor- 
malizes a two-dimensional real pattern 
with respect to its intensity, position, 
size and rotation has been proposed. 

After a ‘“‘center of gravity” of the pattern, 
analogous to that in mechanics, is intro- 
duced, a two-stage coordinate transforma- 
tion, one Cartesian, another polar, is 
performed, yielding a favorable representa- 
tion of the pattern for the normalization. 
Then a finite Fourier transform of the 
pattern represented by polar coordinates 
is taken with respect to the angular variable, 
giving rise to a function-series representa- 
tion of the pattern. Next, a logarithmic 
coordinate conversion is introduced as to 
the radial variable, and an infinite Fourier 
transform in the complex domain is taken, 
giving also a function-series representation. 

The final normalized expression of the 
pattern is obtained by a simple algorithm 
composed of an infinite integral and a 
division involving the above function 
series which vanishes rather rapidly for 
“smooth” patterns for both large series 
numbers and large arguments, and reduces 
to “one” function in case the pattern is 
point symmetrical. 

Lastly, the computational flow-chart for 
the normalization is described and the 
necessary machine time is estimated, 
although approximately. 


Musical Composition and Performance 
by Means of Computing Machines (A; 


April 21, 1960)—H. Isida. (Faculty of 
Science, Tokyo University, Bunkyo-ku, 
Tokyo.) 


Synchronization for Multi-channel PCM 
(I; April 22, 1960)—Y. Nakamaru and 
H. Kaneko. (Nippon Electric Co., 1753 
Tamagawamuko, Shimonuma-gun, Kawa- 
saki-shi.) 


The digital method of synchronization 
is suitable for framing of digital trans- 
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mission channels such as long-frame PCM. 
In this paper a new method of synchroniza- 
tion with an excellent characteristic is 
derived from a theoretical analysis of the 
recovery process. In this system the 
framing-pulse sequence is located in a 
group at the beginning of a frame, and a 
hunting procedure is performed in such 
a way as to reset immediately the channel 
separator at each instant of error detection. 

It is shown that according to this method, 
normal synchronism can usually be restored 
in only one frame of collapse by employing 
ten or more framing pulses in a frame. 
Optimum codes and a method of frame 
composition which provides an optimum 
characteristic are presented. The stability 
of synchronism is highly improved by 
adding a locking circuit. The results of 
the analysis are proved experimentally 
by using an empirically made recovery 
process simulator. 


On the Sequential Design of Experiments 
(A; July 7, 1960—M. Sakaguchi. (Univer- 
sity of Electro-communications, 14 Kojima- 
cho, Chofu-shi, Tokyo-to.) 


On the Application of a Speech Clipper to 
an SSB Radio Telephone System using 
VODAS (I; July 15, 1960)—U. Tsuruoka 
and J. Nakamichi. (Japan’s Overseas Radio 
and Cable System, 1-5 Otemachi, Chiyoda- 
ku, Tokyo.) 


This paper deals with signal-to-noise 
improvements by the use of a speech 
clipper in an SSB radio telephone system 
using the VODAS. The results of the studies 
show: 1) it is possible to clip the speech 
currents 10 db from the peak level at the 
input of the transmitter with little influence 
upon the naturalness of the voice; 2) in the 
application of a speech clipper to a system 
using the VODAS, the intelligibility of 
the system can be improved to be more than 
93 per cent; this figure is that observed 
in one-way transmission, and equivalently 
about ten times the power of the usual 
transmitter can be obtained. 
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